01, The frozen 5 KPIs
RoastIQ scores every ad on the same five KPIs in the same order with the same weights. The names are frozen. The weights are frozen. This is not a stylistic choice, it is the only way a score from March can be compared to a score from October, or one client's ad compared to another's at the same benchmark percentile.
Beat the Skip carries the heaviest weight because in feed environments, TikTok, Reels, Shorts, nothing else matters if the viewer is gone in two seconds. Get Noticed and Brand Impact share the middle weight band. Build Brand sits at 15%, not because long-term equity is unimportant, but because for a single ad it is the slowest signal to move.
02, How the composite is computed
The composite is a weighted sum, not an average. The difference matters when one KPI is dragging.
For our worked example, Beat the Skip 70, Get Noticed 65, Brand Impact 45, Sell Proposition 68, Build Brand 60, the math is:
03, The three verdicts
Three states. One ladder. No half-verdicts.
Scale
Strong skip resilience, strong impact, no hidden weakness. Push spend, build variants, do not over-engineer.
Sharpen
Real strengths, one or two weak spots. The decision trace tells you which KPI to fix and roughly where in the creative.
Rebuild
Structural problems. Editing won't save it. Start over from the brief, not from the cut.
The double rule on Scale, composite ≥ 70 and no KPI below 55, exists to stop a single weak KPI from being averaged away by four strong ones. The double rule on Rebuild does the same on the other side: two or more KPIs under 45 is a Rebuild even if the composite scrapes 56.
04, Confidence labels
Every KPI score comes with a confidence label. It is not a number you can multiply by anything, it is a flag that tells you how much to trust the verdict before you act on it.
Three inputs shape confidence:
- Benchmark density. The score is a percentile against the benchmark pool. If your ad's category has 500+ ads in the pool, confidence is high. If it has 40, the label says so. We never silently downgrade, the trace exposes the cohort size.
- Replicate agreement. The model is run multiple times. If the KPI score drifts more than a small band across replicates, confidence drops.
- Attribute detection certainty. Whether the model is sure the brand logo appeared at 1.2s vs guessing. Detection is roughly 85% accurate, the formal LLM-as-judge inter-rater reliability paper is in pipeline.
05, Reading the decision trace
Every analysis stores a Zod-validated record. This is what makes a RoastIQ verdict auditable instead of vibes-based. A trimmed example:
The fields you cannot skip: model_version (so you can reproduce the run), benchmark_pool_version (so the percentile makes sense), evidence (frame and transcript pointers, not vibes), and recommendations (specific enough to act on).
06, Worked example: composite 62
A 9:16 brand ad for a mid-market beauty client. Eight seconds. Product-led, with a brand logo arriving late at 6.8s. Here is the full KPI table the report returned.
| KPI | Weight | Score | Contribution | Confidence | Band |
|---|---|---|---|---|---|
| Beat the Skip | 25% | 70 | 17.5 | high | Strong |
| Get Noticed | 20% | 65 | 13.0 | high | Moderate |
| Brand Impact | 20% | 45 | 9.0 | medium | Moderate |
| Sell Proposition | 20% | 68 | 13.6 | high | Moderate |
| Build Brand | 15% | 60 | 9.0 | medium | Moderate |
| Composite | 100% | 62 | 62.1 | high | Sharpen |
The verdict is Sharpen. The composite is in the 55–69 band, no KPI is under 45, so it does not trip the Rebuild double rule. The weak spot is unambiguous, Brand Impact at 45, and the trace pinpoints why: the first distinctive brand asset does not appear until 6.8s of an 8-second cut. The recommendation is mechanical: move the asset before 2.0s.
07, Three reviewers, three reads
Before the trace existed, this exact ad went through an internal review. Three people. Three verdicts. Same eight seconds.
Saw a clean hook, a clear product shot, no obvious craft issue. Pattern-matched to ads that performed last quarter. Did not notice the logo timing.
Bothered by the late brand entry and the flat closing card. Felt the whole spot lacked a distinctive idea. Wanted to go back to brief.
Argued the hook and proposition were earning their weight. Suspected the brand asset placement was the issue, but had no way to prove it inside the meeting.
The decision trace did not pick a winner by personality. It picked one by evidence: KPI scores, benchmark percentiles, a frame-level pointer to the 6.8s logo entry, and a recommendation that any editor could execute in twenty minutes. The Strategist was right. The other two were not wrong about what they noticed, they were guessing about what to do next.
08, The honest caveat
RoastIQ scores are model predictions, not measurements. We validate them against public engagement and click-intent signals only, TikTok engagement, TikTok CTR, YouTube view counts. As of 2026-05-05, held-out out-of-sample Spearman ρ is between +0.30 and +0.32 across those outcomes. Pool-wide quintile lift is 6.5×: ads in the top predicted quintile outperform the bottom quintile by that factor on the public outcome signal.
What this guide does not claim:
- That a 62 predicts ROAS, attributed sales, or in-market business outcomes.
- That a 62 predicts brand recall the way a survey would. Different construct.
- That Meta-Feed scores have the same validation depth, for brand ads on Meta, outcome data is sparse and the scores are directional defaults.
What it does claim, plainly: a Sharpen verdict with a Brand Impact score of 45 and a frame-level pointer to a late logo is more decision-useful than three smart people disagreeing in a meeting.
Keep reading
Hook rate vs thumbstop
Two metrics. Two questions. Which one are you actually answering?
Brand cue placement
Where the logo, palette, and proposition land, and why timing beats frequency.
Synthetic vs live test
When a simulated panel is enough and when it isn't.
The first two seconds
What earns the rest of the ad the right to exist.
Benchmark credibility
500 ads × 43 attributes = 21,500 data points. How the pool is built.
All six guides
The full creative analysis hub index.
Frequently asked
What does a composite score of 62 actually mean?
Composite 62 falls inside the Sharpen band (55–69). It means the ad has real strengths but one or two KPIs are pulling the verdict down. The decision trace tells you which KPI and where in the creative to act.
Can the KPI weights ever change?
No. The 5 KPI names and their weights, Beat the Skip 25, Get Noticed 20, Brand Impact 20, Sell Proposition 20, Build Brand 15, are frozen by product rule. Changing them would invalidate historical comparisons and benchmark percentiles.
Why are scores predictions and not measurements?
RoastIQ is a model. We validate predictions against public engagement and click-intent signals from TikTok and YouTube, held-out OOS Spearman ρ of +0.30 to +0.32 as of 2026-05-05. We never claim to predict ROAS, attributed sales, or brand recall.
How do I know the verdict is trustworthy?
Every analysis stores a decision trace: model_version, benchmark_pool_version, KPI scores, evidence pointers, recommendations. Confidence labels reflect benchmark density, replicate agreement, and attribute detection certainty. You can audit any verdict against its evidence.