New research guide: The Confidence Economy, how AI is changing fashion commerce.
Read it →
Elara proprietary
Holdout-Based Lift Report
A holdout-based lift report measures the true incremental impact of an AI tool by randomly assigning some shoppers to a control group with no access to the tool, then comparing their outcomes to the group that had access.
This is an Elara-specific term. The definition below reflects how Elara uses it.
What this means
Most AI shopping tool vendors report "AI-influenced revenue" — revenue from any session that included an AI interaction. This metric overstates true impact because shoppers who choose to use an AI tool are not a random sample. They are typically more engaged and more purchase-intent than shoppers who don't. Attributing all their revenue to the AI tool credits the system for sales that would have happened anyway. A holdout-based lift report corrects for this. A small percentage of eligible shoppers — typically 5-10% — are assigned to a holdout control group. They browse the store normally without access to the AI tool. After a defined period (14-30 days), outcomes for the exposed group are compared to the holdout group: conversion rate, average order value, return rate, and revenue per visitor. The difference between the two groups is the true incremental lift. Revenue that existed in the holdout group would have happened without the AI tool. Only the additional revenue — the delta — is attributable to the system.
Why it matters for fashion brands
A 3x conversion lift on AI-engaged sessions can represent a 10-15% store-level lift if only 5% of shoppers engage with the tool. These two numbers come from the same data but tell very different stories. The holdout report tells you which number is real. Brands that evaluate AI tools without holdout measurement often make investment decisions based on metrics that overstate impact by 2-3x. The holdout-based lift report is the difference between knowing your tool works and believing your vendor's dashboard.
Related terms
Incremental Lift — The actual revenue improvement caused by a tool, isolated through a holdout test. A/B Testing — A controlled experiment splitting traffic between two variants to measure outcomes. Conversion Rate — The percentage of website sessions that result in a purchase.
See how holdout-based measurement works
Understanding this methodology helps you evaluate any AI vendor's lift claims critically.