Decoding a format with no spec, and searching it as you type, on a phone

Notes from building Vela, a player for iPhone and iPad that signs in to the user's own Xtream account. Vela includes no content and no provider; everything below is about what happens to the account's catalog once it reaches the phone.

There is no Xtream specification

There is a widely cloned PHP panel and a decade of forks. The same field is an Int on one provider and a quoted string on the next. tv_archive arrives as 1, "1" or true. Ratings come back as "" when unset. category_id is a string in the stream list and an int in the category list, on the same server.

Decode that strictly with Codable and one odd field kills a response tens of thousands of rows long. So every field is read through a small set of loose* helpers that accept each shape a field has been seen in, and arrays decode through a Failable<T> wrapper, so a single malformed row costs that row rather than the whole catalog.

struct Failable<T: Decodable>: Decodable {
    let value: T?
    init(from decoder: Decoder) throws {
        value = try? T(from: decoder)
    }
}

Two consequences are worth knowing for anyone doing the same:

Synthesised encoding has rules. CodingKeys must name each stored property exactly once and nothing else, or the compiler will not synthesise encode(to:). Wire-only keys (alternate spellings, nested info objects, fields that fold into one property) live in a second private enum, read from its own container.
Pin the date strategy on both sides. The panel sends a bare number as a Unix timestamp, but JSONEncoder's default writes a Date as seconds since 2001. A cache written with the default comes back 31 years early. Pinned to secondsSince1970 both ways, the on-disk cache re-decodes through exactly the same path as a live response.

Search, on the device, while someone types

There is no server, and a provider's catalog is large, so search runs on the phone against the SQLite catalog, in two stages. (Search is part of Vela Pro.)

Candidates: FTS5, every word a prefix

An FTS5 table indexes every title, plus the genre, cast and director of every series, as normalised text. Every query word is treated as a prefix, so harrow va finds Harrow & Vale while it is still being typed.

Ranking: fzf's idea, then structure on top

Only the candidates are scored, with a greedy matcher in the spirit of fzf: bonuses for matching at a word boundary and for consecutive runs, penalties for gaps. Then come structural boosts that deliberately dominate the character-level score: an exact title, a title prefix, every term matched as a whole word. Typing meridian should return the channel called Meridian News, not whichever title happens to have the tightest run of those letters. Length is the final tiebreak, so Meridian News sorts above Meridian News 24 Alternate 4.

Typos, the Typesense way, without Typesense

Typesense is a server, and Vela has none. But what makes it forgiving is a short list of rules with documented defaults, and those carry over. A query that finds nothing as typed has its unknown words replaced by up to four spellings the index does hold: one edit allowed from four letters, two from seven, counted as Damerau–Levenshtein against FTS5's own vocabulary table. If that still finds nothing, words are dropped one at a time, the one carrying the fewest documents first. A corrected match always ranks below one matched as typed.

Groups, each its own query

Results are grouped into series, films, actors, channels, directors and genres, and each kind is its own query. Otherwise thirty thousand channels could crowd the series out of the candidate pool, and the counts would lie. Actors, directors and genres come from splitting the series' free-text fields into separate values; every group opens onto a page with all of it.

What it adds up to

None of this is exotic: lenient decoding, an FTS5 index, a ranker with a clear order of precedence, and a handful of typo rules borrowed from a search engine that documents its defaults. The point is that all of it runs on the phone, so what anyone searches for never leaves it.

Vela Xtream: IPTV Player is on the App Store for iPhone and iPad. It needs an account with an Xtream-compatible provider; it comes with no content of its own.

The other note, on how Vela recommends with no server and no other viewers: Recommendations with no server and no other viewers, on a phone.