What does an iROAS confidence interval actually tell you?

Your iROAS is not 2.4x. It is 0.8x to 4.0x. An iROAS confidence interval is the range of incremental returns your test could not rule out, and that range, not the single number on the slide, is what tells you whether the result is solid enough to move budget on. It still will not tell you whether the channel made money. For that you read the range against two lines: zero, and your breakeven iROAS.

What is an iROAS confidence interval?

It is the range of incremental returns consistent with your test, not a single point. iROAS is the extra revenue your ads caused divided by what you spent. You never see that extra revenue directly, you estimate it, so the answer arrives as a range that shows how much the test could and could not pin down.

Here is the example this post keeps using. A holdout estimates $240,000 of incremental revenue on $100,000 of spend, so the point estimate is 2.4x. Bounds of 1.9x to 2.9x make that result precise. Bounds of 0.8x to 4.0x are the same headline on a completely different decision.

One caveat, because the loose version is everywhere. A confidence interval is not a 90% chance the true value sits in this exact range. It comes from a method that captures the true value 90% of the time when you repeat it, provided the method's assumptions hold. The confidence level belongs to the method, not to the one range in front of you.

The working translation: the point estimate is the test's best guess. The interval is how much it could not pin down.

Platform-attributed ROAS does not belong in this conversation. Ads Manager can tell you a sale happened after an impression. It cannot tell you what would have happened without the ad, and that gap is the whole problem. An error bar around an attributed number does not make it causal. You need an estimate from an experiment or a causal model, with the uncertainty that came with it.

Why isn't 1.0x your breakeven iROAS?

Because iROAS counts revenue, and you pay your bills with margin. At 1.0x, a dollar of ads returned a dollar of revenue. Unless your contribution margin is 100%, that revenue did not cover the product plus the ad. You lost money. Your real breakeven is set by what you keep on each order.

So the lower your margin, the higher the return you need just to break even.

Contribution margin is what survives an order after COGS, fulfillment, shipping, payment fees, returns, and discounts, and before advertising. If ad spend is already out, dividing 1 by it double-counts the ad and hands you a target that is too easy.

Windows have to match. If the iROAS measures first-order revenue, hold it to first-order margin. And the formula assumes the incremental orders carry your average margin, which is worth checking. If your blended margin is 30% but the orders the ads created run at 24%, your real breakeven is 4.2x, not 3.3x.

The arithmetic is trivial. Getting finance and marketing to agree on the number takes three weeks.

Is there a universally good iROAS?

No. Across 225 geo incrementality tests run on Stella's self-service platform, the median was 2.31x, with the middle half between 1.36x and 3.24x. Read the caveat first: these are brands that chose to test, mostly US DTC ecommerce, so it is where tested brands land, not a market average. That spread is across 225 tests, not the interval around any one of them.

Put the median against the markers. At a 50% margin, breakeven is 2.0x and the median clears it. At 35%, breakeven is 2.9x and the median falls short while the top quartile still clears. At 25%, breakeven is 4.0x, and even the 75th-percentile brand sits below the line.

So you cannot call a result strong or weak without knowing what the brand keeps on an incremental dollar. Any benchmark that skips that step is grading against the wrong line. Full methodology is in the study.

Is my iROAS good enough to move budget?

Only if you check it against both lines. A significance test asks whether the interval excludes zero, which tells you whether there was any detectable lift at all. It says nothing about whether that lift paid for itself. Put the interval on an axis with zero and your breakeven iROAS, and six outcomes fall out.

Row three is the one nobody names, so name it. A result can be statistically significant, causally incremental, and financially unprofitable. Take a 2.4x with an interval of 2.0x to 2.8x. It excludes zero, the lift is genuine, the vendor's slide says the test passed. At a 25% margin, breakeven is 4.0x and the whole range sits under it. The channel worked. It lost money. Both are true.

Try it on your own numbers. Enter the bounds your last test reported, then move the margin and leave the result alone.

Row two gets misread just as often, the other way. An interval straddling breakeven is not a channel that failed. It is a test that could not resolve the question you asked, and teams kill channels over it when the honest read is that the design needed more geos. Our geo-testing guide covers what actually widens an interval.

The hurdle is a business input, not a measurement output. Anyone who says a range clears breakeven without asking your margin is quietly using 1.0x.

One thing this rule is not: a law of statistics. Requiring the lower bound to clear breakeven is a conservative policy, the right default for a large or hard-to-reverse move. A small, reversible bet with a point estimate well above breakeven can be worth making first. The interval gives you the uncertainty. Management decides how much of it to buy.

How far can I scale on this result?

Not as far as you think. A holdout estimates the return across the spend it tested, and nothing outside it. If cutting $100,000 of Meta cost you $400,000 of revenue, that supports 4.0x over that slice of budget. It does not mean the next $100,000 returns another $400,000. Response curves bend, and the best customers get bought first.

The size of the cut decides what you measured. Turn a channel off entirely and you get the average return across all its spend. Cut it 15% and you estimate the return over that smaller slice, which is usually closer to the next budget decision. Both get reported as "iROAS." A report that does not state the tested spend has hidden which one it handed you.

So "the interval clears breakeven" means the tested decision was defensible. It does not mean scale without limits. Move in bounded steps and update. Mapping where returns decay takes repeated tests or a calibrated media mix model.

How tight does the interval need to be?

Tight enough to separate the decisions in front of you. There is no universal width. At a 3.0x breakeven, a 6.0x with bounds of 4.2x to 7.8x is wide, and every value in it clears the hurdle. A 3.1x with bounds of 2.9x to 3.3x is narrow, and it lands right on the line it needed to resolve. The wide one is actionable. The narrow one is not.

Precision is not narrowness. Precision is being able to tell your options apart. Width is mostly a property of the design: the confidence level, the inference method, the spend contrast, how many genuinely independent geos you have, and how much your baseline moves on its own. Adding markets that all rise and fall together adds less than the count suggests.

"Run it longer" is not a universal fix either. A longer window invites promotions, seasonality, and competitor moves. More data only helps when it still belongs to the same experiment.

Here is the part most teams miss. Power a test to beat zero and you may show the channel makes revenue while still not knowing whether it pays. At a 4.0x breakeven, a test built to separate 1.0x from zero answers a question you did not ask. The threshold that matters for the budget is often breakeven, not zero, and that belongs in the minimum detectable effect before launch, not the post-mortem.

On reporting: Stella pre-specifies 90% two-sided intervals for operating decisions and does not loosen the threshold after seeing the result. The published 225-test study used 95% bootstrap intervals from 1,000 resamples in 7-day blocks, so weekly patterns are not treated as independent days. Different bars on purpose, because a published analysis carries a stricter one than an operating call.

Can a precise iROAS confidence interval be wrong?

Yes, and a tight interval is where false comfort lives. A confidence interval quantifies the uncertainty the method knows about. It does not know your counterfactual broke, that customers crossed between markets, that spend never changed as designed, or that the model was picked after someone saw which result looked better.

The interval is conditional on the design. A precise answer to a biased test is still a biased answer. So every result needs two passes. Is it precise? Look at the interval. Is it credible? Look at the design. Did the control predict the treatment markets out of sample beforehand? Was the spend change real and verified? Was spillover controlled? Was the analysis plan set before the number came back?

That second pass is what vetting a measurement partner actually means.

A note on MMM. Incrementality tests report confidence intervals. A Bayesian media mix model reports credible intervals, which hold a stated share of posterior probability given the model and its priors. The language differs, the standard does not: both get read against your hurdle, never against 1.0x. A Bayesian model lets you ask it directly, as the probability that a channel beats your breakeven, which is a cleaner input to a budget call than a lower-bound rule.

What should I ask in a measurement review?

Ask three things at once. What are the bounds on this estimate, where does our breakeven iROAS sit inside them, and what spend change did the test actually identify? Each part catches a different failure. A point estimate hides the uncertainty. Clearing breakeven without asking your margin means they used 1.0x. Recommending unlimited scale off one tested cut confuses average return with marginal.

The number is not the deliverable. The defensible budget decision is.

Bring us the result your vendor called a win. We will put its bounds and your real breakeven on the same axis before you move a dollar.

FAQ

What is a good iROAS?

There is no universal one. A result is attractive when it clears your required return after contribution margin, payback window, and risk. For a first-order read, breakeven iROAS is 1 divided by contribution margin. A 2.5x is profitable at a 50% margin and a loss at 25%.

Can a statistically significant channel still lose money?

Yes, and it is the most expensive misread in measurement. Significance says the lift is distinguishable from zero. Profitability asks whether the lift beats your economic hurdle. An interval can sit entirely above zero and entirely below breakeven at the same time.

What does it mean if my iROAS confidence interval includes zero?

The test did not separate a positive effect from no effect at your confidence level. That is not proof the channel does nothing. Check the minimum detectable effect and the upper bound. If the upper bound still holds a return worth having, the test was too imprecise to call.

What is the difference between a confidence interval and a credible interval?

A confidence interval comes from a procedure that captures the true value at a stated rate across repeated applications. A credible interval, which a Bayesian media mix model produces, holds a stated share of posterior probability given the model, the priors, and the data. Different logic. Both get read against your breakeven.

Should I use a 90% or 95% confidence interval?

Pick before you see the result. Stella pre-specifies 90% two-sided for operating decisions. A 95% interval is wider on the same data because it is built to a lower error tolerance, not because the evidence is stronger. Choosing the level after seeing the number turns your threshold into a negotiation.

Can I scale using the lower bound?

Not directly. The lower bound describes uncertainty around the return for the spend contrast you tested. It does not estimate the return on the next dollar. Use it to judge whether the tested decision was sound, then move in bounded steps and update.