ROAS Guarantees
Return on Ad Spend (ROAS), the ratio of value to cost, is a common performance metric for marketing campaigns. The value can be any quantity of interest: gross revenue, net profit, orders, units sold, new users, qualified leads, downloads… whatever KPI we’re looking to push. ROAS sometimes goes by the more general term Return on Investment (ROI), or replaced with its reciprocal (e.g. ACOS/TACOS among Amazon sellers), but it’s always about that ratio.
Calculating ROAS is easy enough for historic, analytically fossilized campaigns. A more meaningful challenge is to hit ROAS targets in our current campaigns: to guarantee, in advance, that we’ll get at least this much value, before we do any of the spending.
Unfortunately, we can’t observe the future — not quite yet. Sans operational crystal balls, we must infer that future.
Conceptually, that’s it. Once we know the campaign’s future value, we can calculate the maximum cost that upholds our ROAS target — the most we’d be willing to pay for that value. The result then informs bidding, target audience selection, or just a simple go/no-go decision, depending on the campaign type.
Inference
To quantify our uncertainty about the future, we map each possible outcome of the campaign to its probability. In other words, we infer distributions of the campaign results.
Let’s make it concrete with a classic pay-per-click example. Each potential outcome is a specific number of impressions, clicks, and (incremental) conversions, with each conversion having its own order value, a specific monetary amount. To keep things simple, for now, let’s work in terms of gross revenue, and ignore things like customer life time value and the present value of future cash flows, channel saturation and organic cannibalization, fractional attribution, budget limits… we’ll touch on them all later.
Since essentially we’re buying clicks, let’s aggregate those potential campaign outcomes into a distribution of the average value per click (VPC). These look something like this:
That’s not a normal distribution, nor does it belong to any other common family. It’s almost always asymmetric (skewed), often multimodal (has more than a single local maxima or “peak”), and presents interesting kurtosis (the “chubbiness” of the tails, the propensity of outliers). Well, interesting to some. We won’t really get into any of the math, except to call things by their name: a conditional posterior predictive and its inverse complementary cumulative.
Making Guarantees
Let’s continue our example with a 5:1 ROAS target, meaning we’d like at least $5 in gross revenue for every $1 we spend through our campaign (ACOS no greater than 20%). In other words, we want to make sure our campaign will cost at most a fifth of the revenue it generates.
Our inference tells us that there’s a solid 90% chance that the average value of a click will be $3.4963 or more. If we paid a fifth of that to get the click — if our average cost per click was $0.6993 — then we’d have exactly that solid 90% chance of hitting or surpassing our 5:1 ROAS target; naturally, that chance can only go up if we paid less.
We got ourselves a strong ROAS guarantee: as long as we keep the average cost per click at or below the threshold, we have at least a 90% chance of success. Crucially, the guarantee holds before we start the campaign — we could use it in a planning meeting to get the budget for the campaign approved, for example.
Guarantee Levels
Perhaps we’d like to be more conservative, have a stronger guarantee of hitting our ROAS target; this will push our maximum average cost per click lower, driving less clicks and conversions, so our total revenue should decrease as well — we’d be “playing it safer” by spending less and making less, but with a stronger guarantee of hitting our ROAS target. Or perhaps we’d like to be more aggressive, take on a little more risk, in exchange for potentially more clicks and conversions. We can adjust that 90% to any credible lower bound of the value per click:
Once we make our choice, the maximum average cost per click that guarantees our 5:1 ROAS target is a fifth of the corresponding value, just like above:
So we can have our ROAS guarantees as strong as we like. And we can also go in the opposite direction: for any average cost per click, calculate the probability of hitting the ROAS target if we paid that much.
At the conservative extreme, we can have a 99.9% chance for our 5:1 ROAS target if we paid, on average, at most $0.5972 per click. Of course, we might only get very few clicks at this low price, or none at all — our inference tells us that too — but if we can only tolerate a one-in-a-thousand chance of missing our ROAS target, then we really shouldn’t be paying more.
On the other hand, we wouldn’t normally go for guarantees weaker than 50%. The value per click is as likely to be above $3.8793 as below (the median); paying over $0.7759 per click makes missing our 5:1 ROAS target more likely than hitting it. In fact, the expected value per click — the mean of the distribution — is even higher, as value distributions are almost always positively skewed. That’s one reason why optimizations based on “averages” fail to deliver so often: their chance of success is usually worse than a coin toss.
Everything we’ve done so far just reifies the general principle from the introduction: knowing the future value allows us to calculate the maximum cost that guarantees our ROAS target. There’s uncertainty in our knowledge of the future — it’s a distribution, not a single point value — so we get a family of those, one at each level of guarantee. If our value distribution was an infinitely narrow “needle” of zero variance, all levels of guarantee would have converged on the exact same maximum average cost per click; the wider the distribution, the bigger the spread.
But limiting ourselves to such a singular view can lead to lukewarm results, leaving much of the campaign’s potential on the table. For better performance, we need segmentation.
Deconfluencing
A marketing campaign is an amalgam of different ads, keywords and their match types, landing pages, target audience demographics and cohorts, contextual attributes, … all interacting to produce its outcomes. Here are the value per click distributions for three segments of our exmaple campaign, each with its own 90% lower bound. For comparison, the distribution of the combined VPC (in dashing gray) is exactly the same as in the first plot above:
The clicks of segment A are much more valuable than those of the other two segments: at our 90% guarantee level, segment A brings in at least $3.7329 in gross revenue per click, versus $2.7394 for segment B and just $2.5437 for segment C.
The three segments could be three groups of keywords. Or traffic from desktop/laptop computers versus mobile devices versus tablets. Or clicks during business hours, workweek evenings and weekends, high/medium/low household income zip codes, new versus repeat customers versus inactive past customers (last purchase more than X days in the past) — or any combination of those.
Composition
Note how the distributions for the individual segments are all quite a bit “wider” than the global one. Since there's less data for each segment on its own than for all lumped together, even with hierarchical modelling (partial pooling) we end up with more variance. This is especially evident for segment C, which makes up only about 9% of the campaign. The combined distribution captures our uncertainty regarding how impressions, clicks and conversions get split between the underlying segments, as well as our uncertainty regarding the performance of each.
That said, the plot is somewhat misleading — the three segments are highly correlated, meaning we’re much more likely to observe values at either low or high end for all segments, than to get values at one end for some segments and at the other end for the rest. That’s partly becasue many of the underlying performance factors are common to all segments: if the checkout page takes forever to load, the conversion rate will plummet for all keywords and ads, regardless of the user’s device type, geographic location, age, purchase history, or their favorite color. Unfortunately, a more truthful representation of the joint distribution has too many dimensions to plot on this screen.
Since any segment that holds more than a single entity can be further segmented, those observations hold true recursively down the lattice.
Segmented Guarantees
Let’s apply the same logic we used above, only now we do it for each segment: find the 90% lower bound of the value per click, and have our maximum cost be at most a fifth of that value.
For a 90% guarantee of hitting or surpassing our 5:1 ROAS target in each of the individual segments, the maximum average cost per click should be $0.7466 in segment A, $0.5479 in segment B, and $0.5087 in segment C. None of those numbers equals that $0.6993 we calculated previously for the combined traffic, and for a good reason: a different value means a different maximum cost guarantees our target ROAS, that’s just our general principle in action.
But these are “local” guarantees, holding for each segment on its own, independent of the others. Often, we don’t really care about the ROAS of the individual segments, so long as we can guarantee that overall we get our ROAS target, looking at the entire campaign as a whole. What we want is a “global” guarantee, and we’ll see later that this more relaxed guarantee can greatly improve the expected performance of our campaign. To get it, we need to know how our maximum average cost per click impacts the number of clicks we’ll have in each segment — in practice, this usually involves an auction mechanism, so we should discuss those first.
Bidding
Let’s put segmentation aside for a minute, and go back to our first, unsegmented 5:1 ROAS guarantee. How would we use it in practice?
We can’t simply fix the bid or max CPC setting in our campaign to that $0.6993 cost per click, for a number of reasons:
-
Many platforms implement some sort of an auction mechanism, often a variant of generalized 2nd price, which makes our inference dependent on that bid or max CPC setting. In other words, the auction mechanism means there just isn’t a single distribution like we inferred above — instead, we need to infer one for each bid or max CPC setting.
A higher bid or max CPC setting can drive more impressions at more prominent parts of the page, where users are more likely to click; conversion rates and order value statistics often change with the placement (position) of ads, not always in the same direction — and all of those influence the value of our campaign. The actual cost of the clicks is a function of all competing bids, ours included, among other factors. Intuitively, for a ROAS guarantee, we need to take all that into account.
Similar reasoning applies for the 1st price auction of many display campaigns, or discounts and audience selection in sell-through email campaigns, or loyalty program rewards — whenever our settings influence value and/or cost, ROAS guarantees become sensitive to those settings.
-
On most platforms, a bid or max CPC setting applies to individual clicks, yet our 90% guarantee holds for a maximum average cost per click. If we ignored the difference, we’d usually end up much more conservative than we intended to be.
Imagine our campaign could generate for us at most two clicks, at a cost of $0.65 for one and $0.71 for the other. If we bid high enough to get both clicks, our average cost per click will be $0.68 — well below the 90% guarantee maximum cost of $0.6993, so we should be happy to bid that high. But if instead we limited the individual cost per click to be at most $0.6993, we’d give up half the traffic: lose the $0.71 click, along with its potential conversion and revenue.
With only the cheaper click, our average cost per click will be $0.65, and we’ll have about 98% chance to get our 5:1 ROAS target or better. That’s much more conservative than the 90% we wanted, at a very high cost.
-
Finally, the platforms may allow only whole cents in a bid or max CPC setting. The quantization means we can’t easily make guarantees at any level of our choosing; we’re limited to only those that correspond to whole cent settings.
So we need a more nuanced type of guarantee, one that depends on our settings. Naturally, if more than a single bid or max CPC setting guarantees our 5:1 ROAS target, we’ll also need to a way to select the best one to use — let’s create the guarantee first, and then go on to optimize our campaign.
Auction Guarantees
For each bid or max CPC setting, we’ll infer its own joint distribution of total value and total cost — the two are strongly correlated — reduced into a distribution of their ratio, that is the distribution of the ROAS for that bid or max CPC setting. This gives us the probability of hitting our 5:1 ROAS target for each setting: if equal to or greater than 90% we have our guarantee, and that bid or max CPC setting becomes “admissible”, eligible for use in the campaign; the others are deemed too risky.
A good way to visualize it is to plot each bid or max CPC setting against its 90% guaranteed ROAS:
A bid or max CPC setting of $0.35 gets us a 90% guaranteed ROAS of at least $7.23 in gross revenue for every $1 we spend in the campaign; clearly, the probability for at least our 5:1 ROAS target must be higher, about 99% in this case — very admissible. Conversely, $0.80 gets us a 90% guarantee of just $4.67 for every dollar we spend, not good enough; the probability of hitting our 5:1 ROAS target is only about 66%, making it too risky.
We got our new strong ROAS guarantee: as long as we set our bid or max CPC anywhere between $0.28 and $0.74 (the admissible set), there’s at least 90% chance that we’ll get our 5:1 ROAS target or better.
The ROAS Plot
Let’s take a couple of paragraphs to understand the shape of that plot. Focusing on the right tail, a higher bid or max CPC setting will likely generate a higher average cost per click; at some point, those rising costs will have less than a 90% chance to fall below a fifth of the average value per click, so the points drop below the admissible threshold. Other factors include the interaction of the campaign segments (next section), often some decline in conversion rates at higher bid levels, and a general increased uncertainty for settings with less or no historical data.
On the left, a lower bid or max CPC setting will likely generate fewer clicks and conversions; below a certain point, it also lowers the probability of hitting our ROAS target, despite the lower average cost per click, due to increased variances in both value and cost distributions. To illustrate, consider a game of flipping a fair coin where we win $1 for every head and lose $0.99 for every tail, and suppose we wanted a 90% guarantee that we’ll make some money by the end of the game. If we only flipped the coin once, the probability of success is 50%, yet it approaches 100% as we continue to play — our guarantee holds only if we play enough rounds of the game. This effect becomes more prominent with lower click counts: higher levels of guarantee, lower conversion rates, smaller audiences (a natural consequence of segmentation), and shorter prediction windows.
What if we chose another guarantee level?
A weaker guarantee admits more bids or max CPC settings. If we’re optimizing for traffic, number of conversions or total gross revenue, we’ll usually end up picking the highest admissible bid or max CPC setting, so lowering the guarantee level would make a difference. In other cases, extending the admissible set often keeps our optimal bid or max CPC setting unchanged; common examples include optimizing for “marketing contribution” (the gross revenue attributed to the campaign minus its cost), and almost all bottom line impacts.
With a stronger guarantee, we may end up with no bids or max CPC settings that guarantee our ROAS target. But even when there isn’t a single admissible value for the entire campaign, that’s not necessarily true for the segments: with a different bid or max CPC setting in each segment, their combined performance may very well reach 90% chance to get our 5:1 ROAS target.
Segmentation
For each bid or max CPC setting, the segments have their own joint value and cost distributions, and so their own unique ROAS plots and admissible sets:
This gets us strong “local” guarantees for each of the segments. For segment A, independent of the other segments, any bid or max CPC setting from $0.41 through $0.82 gives us at least 90% chance for a 5:1 ROAS or better in that segment. For segment B it’s $0.32 through $0.59, and $0.34 through $0.57 for segment C. Obvisouly, all three are very different from our guarantee without segmentation: a bid or max CPC setting of $0.35 for segment A gets us only a 68% chance for our 5:1 target ROAS in that segment, compared to 99% when applied for the entire campaign. Similarly, $0.70 gets us only a 69% chance of success in segment B and 65% in segment C, compared to 97% globally. On the other hand, a bid or max CPC setting of $0.80 is perfectly admissible for segment A, with a 93% chance to get our 5:1 ROAS target in that segment, yet it’s too risky to use across the entire campaign (66%) or for the other segments (21% and 33%).
But we can actually go a step further now: calculate the probability of getting our 5:1 ROAS target in the campaign overall, for any combination of bids or max CPC settings in the individual segments. As usual, any combination that gives us at least 90% chance of success becomes admissible, and the rest ruled out as too risky. Becasue the segment performances are highly correlated, that probability is not the product of the probabilities of success for each segment on its own — in practice, it’s always much higher; however, even if all bids or max CPC settings are “locally” admissible in their respective segment, the 90% guaranteed combined performance may fall below our ROAS target.
With just three segments, we can plot the admissble set. To make it visually clearer, we include only the boundary points that envelope the set, and add a little jitter to circumvent Moiré artifacts:
And so we get our strong “global” guarantee with segmentation. Any bid or max CPC combination inside that wacky volume is admissible — we’ll get at least 90% chance for our 5:1 ROAS target overall if we use it in our campaign. We can easily generalize this guarantee to any number of segments; it’s harder to visualize above three dimensions, but just as easy to use.
This “global” guarantee is much more efficient than the “local” guarantees, since it admits many more combinations. For example, if we set our bid or max CPC to $0.85 in segment A, $0.65 in B and $0.60 in C, we’ll have a 95.64% chance to get our 5:1 ROAS target — more than 90%, making it admissible — even though every single one of those values is outside the range of the “local” guarantee for the segment. With more admissible options to choose from, our optimization is free to pick one that offers better performance, under the same guarantee.
Optimization
For a given segmentation, including the one where we pile everything into a single segment, optimization is very simple: find the bids or max CPC settings from the admissible set that maximize the expected value of the campaign. The ROAS guarantee is a constraint, filtering out any option with over 10% chance to drop below our 5:1 ROAS target; we then just pick the best of the remaining options.
Obviously, slicing and dicing our campaign into different segmentations can make a huge difference. Let’s optimize our campaign for gross revenue: with a single segment (the entire combined traffic), the optimal bid or max CPC is $0.74 — the highest admissible, no surprise — which gets us $22,626 in expected total gross revenue. Optimizing with our three segments drives segment A up to $0.91, and lowers segment B to $0.67 and segment C to $0.58, for an expected total gross revenue of $25,577 — that’s over 13% increase in revenue! Both settings gave us a 90% guarantee for a 5:1 ROAS or better, but segmentation allows us to bid much more efficiently on clicks of different value, for significantly better performance.
A “deficient” segmentation is double whammy inefficient: for a large portion of the traffic we bid too high, performing worse than our ROAS target; for another large portion of the traffic we bid too low, losing out on traffic that would have met our performance goals, sending it over to the competition. Our ROAS guarantee holds anyway — some segments subsidize others, so overall we hit our target with that 90% chance — but a good segmentation allows us to callibrate the bids much better, maximizing the total expected value of the campaign.
Optimal Segmentation
Shouldn’t we segment our campaign further then?
Maybe, since segmentation does come at a price. Roughly speaking, less data in each individual segment means more uncertainty regarding their performance, and more segments means more uncertainty regarding how traffic gets split between them. As we continue to segment our campaign into finer partitions, our ability to make ROAS guarantees diminishes, until at some point we’re constrained into worsening expected performance. A more complete explanation requires that we get into the math, but the intuition is that it’s a balancing act: further segmentation allows higher resolution optimization over a smaller admissible space, where the benefits from the increased efficiency may be outweighed by the restricted domain.
Ultimately, we’re looking for the optimal segmentation: in our campaign, it’s the one that gets us the highest expected gross revenue when we optimize our bids or max CPC settings constrained to a 90% guaranteed 5:1 ROAS.
Actionale & Differentiating
There are many (many!) ways to segment a campaign, not all of them meaningful in our search.
We should only consider actionable segmentations, supported by the channel — if we can’t set a different bid or max CPC for different device types, geographic locations, or new versus repeat customers, then those are irrelevant for optimization in this channel. Bid adjustments, also called modifiers or multipliers, are just a roundabout way of altering bids in distinct segments (except some platforms limit how big a difference we can set); negative keywords and other exclusions effectively set a zero bid for their target.
Next, only differentiating segments can increase the efficiency of our bids: those that have contrastive, dissimilar enough inferred distributions, so that a different bid or max CPC in each would give us better performance than a single common one. There’s little value in breaking up groups of keywords or ads with near identical behavior. Age and gender can have a strong influence over the value of a click if we’re selling apparel, maybe not as much for organic fertilizers or office supplies. We don’t need to guess: our inference gives us those insights directly.
Other Campaigns, Other KPIs
This general approach applies to any type of campaign, optimizing for any KPI.
If we’re paying for impressions, we’ll infer value per impression distributions (instead of per click) — we add the math for modeling the click-through rate of the different segments since our funnel starts earlier now, and we may want to optimize frequency caps on top of the bids, but otherwise we follow the same logic. More generally, whatever we’re paying for, we’ll infer the distributions of the value per that thing — transactions, leads, emails with varying discount codes, …
We can work in terms of net profit instead of gross revenue. Suppose our 5:1 ROAS target was based on a 20% average margin rate (so we’re guaranteeing break-even or better), and let’s optimize for “bottom-line marketing contribution” by subtracting the expected cost of our campaign from the expected net profit it will generate. Our inference already gives us all the relevant distributions, and we get a very different set of optimal bids: $0.72, $0.52 and $0.48 for the three segments. Compared to the optimization from before, we’ll generate only $19,573 in gross revenue instead of $25,577 but at a cost of just $3,006 instead of $4,644 which almost doubles our net contribution, from $472 to $908. Better yet, we can use the actual profit data of the individual historic transactions to infer margin rate distributions in the different segments — different products may have different margin rates, geographic location may impact shipping costs, et cetera — and allow optimal segmentation to take such differences into account. Use your data!
Life-time value, subscription business models, seasonality, incrementality (fractional attribution, organic cannibalization) and other refinements involve changes to how we make our inferences. The math gets slightly more complicated, but the general principle remains unchanged: quntifying our uncertainty regarding the future value gives us strong performance guarantees, before the campaign starts.
K2DQ
Take advantage of K2DQ’s cost effective solutions for marketing teams and agencies. Let’s talk :)