Skip to main content
RoastIQBuyerLensHugoPricingBlogAbout
Client accessDiscuss your campaign
Explore the knowledge hubs

Validation methodology

What our validation measures.

Model validation asks whether a defined prediction relates to a defined outcome on a specified cohort. It is separate from the practical question of which peer group fits your creative.

By Oussama Nakhil · Updated 28 September 2026

Name the model and the outcome

The results below describe the platform-native attribute model. They do not validate the five-KPI composite displayed beside it in reports. The supported outcomes are TikTok click-through rate and YouTube view counts, not sales, ROAS, attributed conversion or consumer recall.

Source snapshot: internal validation evidence documented in August 2026. These are dated research cohorts, not a live count of the entire reference library.

Read held-out and fitted results separately

TikTok CTR

+0.33 on TikTok click-through rate (n=691), held out. On the cohort its weights were tuned on the same model scores +0.43, brand-clustered 95% CI [+0.32, +0.53].

691 ads across 81 brands in the documented cohort.

YouTube views

+0.40 on YouTube view counts (n=397), held out. On the cohort its weights were tuned on the same model scores +0.49, brand-clustered 95% CI [+0.37, +0.58].

397 ads across 72 brands in the documented cohort.

Held-out evaluation measures performance on examples excluded from fitting in that evaluation. Fitted results use the cohort on which model weights were tuned and can be optimistic. The displayed confidence intervals belong to the fitted estimate. No held-out interval is established in this evidence set.

Account for shared brands

Ads from one brand share creative assets and campaign conditions. Treating every ad as independent can make uncertainty look smaller than it is. The fitted intervals above use brand-clustered resampling.

A pool-wide relationship does not prove performance within every country, format or brand. Sample size and coverage must travel with any narrower claim. Meta remains directional where outcome validation is unavailable.

Keep an evidence trail

  • Record the outcome definition, collection window, platform and cohort exclusions.
  • Retain model and benchmark versions with every result.
  • Separate training, fitted evaluation and held-out evaluation.
  • Document missing outcome data and subgroup coverage.
  • Do not turn correlation into a causal improvement claim.

The former volume and category claims on this page have been removed where a publication-ready supporting study was not established. Counting attributes does not increase the number of independent ads or establish discrimination between score bands.

Review the methodology

The detailed methodology and research notes are available in our methodology overview and the annotation reliability discussion. Before relying on a quantitative claim in procurement, request its dated cohort definition and evaluation record.

For using a report in a creative meeting, see the practical benchmark guide. For your own campaign, distinguish AI diagnostics, simulated interviews, expert review and measured customer outcomes.