ROAS Guarantees

Return on Ad Spend (ROAS), the ratio of value to cost, is a common performance metric for marketing campaigns. The value can be any quantity of interest: gross revenue, net profit, orders, units sold, new users, qualified leads, downloads… whatever KPI we’re looking to push. ROAS sometimes goes by the more general term Return on Investment (ROI), or replaced with its reciprocal (e.g. ACOS/TACOS among Amazon sellers), but it’s always about that ratio.

Calculating ROAS is easy enough for historic, analytically fossilized campaigns. A more meaningful challenge is to hit ROAS targets in our current campaigns: to guarantee, in advance, that we’ll get at least this much value, before we do any of the spending.

Unfortunately, we can’t observe the future — not quite yet. Sans operational crystal balls, we must infer that future.

Conceptually, that’s it. Once we know the campaign’s future value, we can calculate the maximum cost that upholds our ROAS target — the most we’d be willing to pay for that value. The result then informs bidding, target audience selection, or just a simple go/no-go decision, depending on the campaign type.

Inference

To quantify our uncertainty about the future, we map each possible outcome of the campaign to its probability. In other words, we infer distributions of the campaign results.

Let’s make it concrete with a classic pay-per-click example. Each potential outcome is a specific number of impressions, clicks, and (incremental) conversions, with each conversion having its own order value, a specific monetary amount. To keep things simple, for now, let’s work in terms of gross revenue, and ignore things like customer life time value and the present value of future cash flows, channel saturation and organic cannibalization, fractional attribution, budget limits… we’ll touch on them all later.

Since essentially we’re buying clicks, let’s aggregate those potential campaign outcomes into a distribution of the average value per click (VPC). These look something like this:

That’s not a normal distribution, nor does it belong to any other common family. It’s almost always asymmetric (skewed), often multimodal (has more than a single local maxima or “peak”), and presents interesting kurtosis (the “chubbiness” of the tails, the propensity of outliers). Well, interesting to some. We won’t really get into any of the math, except to call things by their name: a conditional posterior predictive and its inverse complementary cumulative.

Making Guarantees

Let’s continue our example with a 5:1 ROAS target, meaning we’d like at least $5 in gross revenue for every $1 we spend through our campaign (ACOS no greater than 20%). In other words, we want to make sure our campaign will cost at most a fifth of the revenue it generates.

Our inference tells us there’s a solid 90% chance that the average value of a click will be $3.4963 or more. If we paid a fifth of that to get the click — if our average cost per click was $0.6993 — then we’d have exactly that solid 90% chance of hitting or surpassing our 5:1 ROAS target. Naturally, that chance can only go up if we paid less…

We got ourselves a strong ROAS guarantee: as long as we keep the average cost per click at or below the threshold, we have at least a 90% chance of success. Crucially, the guarantee holds before we start the campaign — we could present it in a planning meeting to approve the campaign budget, for example.

Guarantee Levels

Perhaps we’d like to be more conservative, have a stronger guarantee of hitting our ROAS target; this will push our maximum average cost per click lower, driving less clicks and conversions, so our total revenue should decrease as well — we’d be “playing it safer” by spending less and making less, but with a stronger guarantee of hitting our ROAS target. Or perhaps we’d like to be more aggressive, take on a little more risk, in exchange for potentially more clicks and conversions. We can adjust that 90% to any credible lower bound of the value per click:

At the conservative extreme, there’s a 99.9% chance the average value of a click will be at least $2.9859 — so a 99.9% chance for our 5:1 ROAS target if we paid, on average, at most $0.5972 per click. Of course, we might only get very few clicks at this low price, or none at all — our inference tells us that too — but if we can only tolerate a one-in-a-thousand chance of missing our ROAS target, then we really shouldn’t be paying more. We can have a less extreme, yet still very conservative 99% guarantee if our average cost per click was at most $0.6394, or a 95% guarantee if we paid at most $0.6781 per click, on average.

On the other hand, we wouldn’t normally go for guarantees weaker than 50%. The value per click is as likely to be above $3.8793 as below (the median); paying over $0.7759 per click makes missing our 5:1 ROAS target more likely than hitting it. In fact, the expected value per click — the mean of the distribution — is even higher, as value distributions are almost always positively skewed. That’s one reason why optimizations based on “averages” fail to deliver so often: their chance of success is usually worse than a coin toss.

Everything we’ve done so far just reifies the general principle from the introduction: knowing the future value allows us to calculate the maximum cost that guarantees our ROAS target. There’s uncertainty in our knowledge of the future — it’s a distribution, not a single point value — so we get a family of those maximum costs, one at each level of guarantee. If our value distribution was an infinitely narrow “needle” of zero variance, all levels of guarantee would have converged on the exact same maximum average cost per click; the wider the distribution, the bigger the spread.

But limiting ourselves to such a singular view can lead to lukewarm results, leaving much of the campaign’s potential on the table. For better performance, we need segmentation.

Deconfluencing

A marketing campaign is an amalgam of different ads, keywords and their match types, landing pages, target audience demographics and cohorts, contextual attributes, … all interacting to produce its outcomes. Here are the value per click distributions for three segments of our exmaple campaign, each with its own 90% lower bound. For comparison, the distribution of the combined value per click (in dashing gray) is exactly the same as in the two plots above:

The clicks of segment A are much more valuable than those of the other two segments: at our 90% guarantee level, segment A brings in at least $3.7329 in gross revenue per click, versus $2.7394 for segment B and just $2.5437 for segment C.

The three segments could be three groups of keywords. Or traffic from desktop/laptop computers versus mobile devices versus tablets. Or clicks during business hours, workweek evenings and weekends, high/medium/low household income zip codes, new versus repeat customers versus inactive past customers (last purchase more than some number of days in the past) — or any combination of those.

Composition

Note how the distributions for the individual segments are all quite a bit “wider” than the global one. Since there's less data for each segment on its own than for all lumped together, even with hierarchical modelling (partial pooling) we end up with more variance. This is especially evident for segment C, which makes up only about 9% of the campaign’s future clicks. The combined distribution captures our uncertainty regarding how impressions, clicks and conversions will get split among the underlying segments, as well as our uncertainty regarding the performance of each.

That said, the plot is somewhat misleading — the three segments are highly correlated, meaning we’re much more likely to observe values at either low or high end for all segments, than to get values at one end for some segments and at the other end for the rest. Many of the underlying performance factors are common to all segments: if the checkout page takes forever to load, the conversion rate will plummet for all keywords and ads, regardless of the user’s device type, geographic location, age, purchase history, or their favorite color. Unfortunately, a more truthful representation (the joint distribution) has too many dimensions to plot on this screen.

Since any segment that holds more than a single entity can be further segmented, those observations hold true recursively down the lattice.

Segmented Guarantees

Let’s apply the same logic we used above, only now we do it for each segment: find the 90% lower bound of the value per click, and ensure our maximum average cost is at most a fifth of that value.

For a 90% guarantee of hitting or surpassing our 5:1 ROAS target in each of the individual segments, the maximum average cost per click should be $0.7466 in segment A, $0.5479 in segment B, and $0.5087 in segment C. None of those numbers equals that $0.6993 we calculated previously for the combined traffic, and for a good reason: a different value means a different maximum cost guarantees our target ROAS, that’s just our general principle in action.

But these are “local” guarantees, holding for each segment on its own, independent of the others. Often, we don’t really care about the ROAS of the individual segments, so long as we can guarantee that overall we get our ROAS target, looking at the entire campaign as a whole. What we want is a “global” guarantee, and we’ll see later that it can greatly improve the expected performance of our campaign. To get it, we need to know how our maximum average cost per click impacts the number of clicks we’ll have in each segment — in practice, this usually involves an auction mechanism, so we should discuss those first.

Bidding

Let’s put segmentation aside for a minute, and go back to our first, unsegmented 5:1 ROAS guarantee. How would we use it in practice?

We can’t simply fix the bid or max CPC setting in our campaign at $0.6993 for a number of reasons:

Many platforms implement some sort of an auction mechanism, often a variant of generalized 2nd price, which makes our inference dependent on that bid or max CPC setting. In other words, the auction mechanism means there just isn’t a single distribution like we inferred above — instead, we need to infer one for each bid or max CPC setting.
A higher bid or max CPC setting can drive more impressions at more prominent parts of the page, where users are more likely to click; conversion rates and order value statistics often change with the placement (position) of ads, not always in the same direction — and all of those influence the value of our campaign. The actual cost of the clicks is a function of all competing bids, ours included, among other factors. Intuitively, for a ROAS guarantee, we need to take it all into account.

Similar reasoning applies for 1st price auction in display campaigns, or discounts and audience selection in sell-through email campaigns, or loyalty program rewards — whenever our settings influence value or cost, ROAS guarantees become sensitive to such settings.
On most platforms, a bid or max CPC setting applies to individual clicks, yet our 90% guarantee holds for a maximum average cost per click. If we ignored the difference, we’d usually end up much more conservative than we intended to be.
Imagine our campaign could generate for us at most two clicks, at a cost of $0.65 for one and $0.71 for the other. If we bid high enough to get both clicks, our average cost per click will be $0.68 — well below the 90% guarantee maximum cost of $0.6993, so we should be happy to bid that high. But if instead we limited the individual cost per click to be at most $0.6993, we’d give up half the traffic: lose the $0.71 click, along with its potential conversion and revenue.
With only the cheaper click, our average cost per click will be $0.65, and we’ll have about 98.4% chance to get our 5:1 ROAS target or better. That’s much more conservative than the 90% we wanted, at a very high cost.
Finally, the platforms may allow only whole cents in a bid or max CPC setting. The quantization means we can’t easily make guarantees at any level of our choosing; we’re limited to only those that correspond to whole cent settings.

So we need a more nuanced type of guarantee, one that depends on our settings. Naturally, if more than a single bid or max CPC setting guarantees our 5:1 ROAS target, we’ll also need to a way to select the best one to use — let’s create the guarantee first, and then go on to optimize our campaign.

Auction Guarantees

For each bid or max CPC setting, we’ll infer its own joint distribution of total value and total cost — the two are strongly correlated — reduced into a distribution of their ratio, that is the distribution of the ROAS for that bid or max CPC setting. This gives us the probability of hitting our 5:1 ROAS target for each setting: if equal to or greater than 90% we have our guarantee, and that bid or max CPC setting becomes “admissible”, eligible for use in the campaign; the others are deemed too risky.

A good way to visualize it is to plot each bid or max CPC setting against its 90% guaranteed ROAS:

A bid or max CPC setting of $0.35 gets us a 90% guaranteed ROAS of at least $7.23 in gross revenue for every $1 we spend in the campaign; clearly, the probability for at least our 5:1 ROAS target must be higher, about 99.2% in this case — very admissible. Conversely, $0.80 gets us a 90% guarantee of just $4.67 for every dollar we spend, not good enough; the probability of hitting our 5:1 ROAS target is only about 66.1%, making it too risky.

We got our new strong ROAS guarantee: as long as we set our bid or max CPC anywhere between $0.28 and $0.74 (the admissible set), there’s at least 90% chance that we’ll get our 5:1 ROAS target or better.

The ROAS Plot

Let’s take a couple of paragraphs to understand the shape of that plot. Focusing on the right tail, a higher bid or max CPC setting will likely generate a higher average cost per click; at some point, those rising costs will have less than a 90% chance to fall below a fifth of the average value per click, so we drop below the admissible threshold. Other factors include the interaction of the campaign segments (next section), often some decline in conversion rates at higher bid levels, and a general increased uncertainty for settings with less or no historical data.

On the left, a lower bid or max CPC setting will likely generate fewer clicks and conversions; below a certain point, it also lowers the probability of hitting our ROAS target, despite the lower average cost per click, due to increased variances in both value and cost distributions. To illustrate, consider a game of flipping a fair coin where we win $1 for every head and lose $0.99 for every tail, and suppose we wanted a 90% guarantee that we’ll make some money by the end of the game. If we only flipped the coin once, the probability of success is 50%, yet it approaches 100% as we continue to play — our guarantee holds only if we play enough rounds of the game. This effect becomes more prominent with lower click counts: higher levels of guarantee, lower conversion rates, smaller audiences (a natural consequence of segmentation), and shorter prediction windows.

What if we chose another guarantee level?

A weaker guarantee admits more bids or max CPC settings. If we’re optimizing for traffic, conversions count or total gross revenue, we’ll usually end up picking the highest admissible bid or max CPC setting, so lowering the guarantee level would make a difference. In other cases, it may have no impact on our optimal bid or max CPC setting; common examples include optimizing for “marketing contribution” (the gross revenue attributed to the campaign minus its cost) and almost all bottom line targets.

With a stronger guarantee, we may end up with no bids or max CPC settings that guarantee our ROAS target. But even when there isn’t a single admissible value for the entire campaign, that’s not necessarily true for its segments: with a different bid or max CPC setting in each segment, their combined performance may very well reach 90% chance to get our 5:1 ROAS target, so…

Segmentation

For each bid or max CPC setting, the segments have their own joint value and cost distributions, implying their own unique ROAS plots and admissible sets:

This gets us strong “local” guarantees for each of the segments. For segment A, independent of the other segments, any bid or max CPC setting from $0.41 through $0.82 gives us at least 90% chance for a 5:1 ROAS or better in that segment. For segment B it’s $0.32 through $0.59, and $0.34 through $0.57 for segment C.

Again, none of the three “local” guarantees is the same as the guarantee for the combined traffic: a bid or max CPC setting of $0.35 in segment A gets us only a 68% chance for our 5:1 target ROAS in that segment, compared to 99% when applied in the entire campaign. Similarly, $0.70 gets us only a 69% chance of success in segment B and 65% in segment C, compared to 97% globally. On the other hand, a bid or max CPC setting of $0.80 is perfectly admissible for segment A, with a 93% chance to get our 5:1 ROAS target in that segment, yet it’s too risky to use across the entire campaign (66%) or in the other two segments (21% and 33%).

But we can actually go a step further now: calculate the probability of getting our 5:1 ROAS target in the campaign overall, for any combination of bids or max CPC settings in the individual segments. As usual, any combination that gives us at least 90% chance of success becomes admissible, and the rest ruled out as too risky. Becasue the segment performances are highly correlated, that probability is not the product of the probabilities of success for each segment on its own — in practice, it’s always much higher. Nevertheless, even if all bids or max CPC settings are “locally” admissible in their respective segments, the 90% guaranteed combined performance may fall below our ROAS target.

With just three segments in our example, we can actually plot this admissble set. To make it visually clearer, we include only the boundary points that envelope the set, and add a little jitter to circumvent Moiré artifacts:

And so, finally, we get our strong “global” guarantee with segmentation. Any bid or max CPC combination inside that wacky volume is admissible — we’ll get at least 90% chance for our 5:1 ROAS target overall if we use it in our campaign. Of course, we can generalize this to any number of segments; it’s harder to visualize above three dimensions, but just as easy to use.

This “global” guarantee is much more efficient than the “local” guarantees, since it admits many more combinations. For example, if we set our bid or max CPC to $0.85 in segment A, $0.65 in B and $0.60 in C, we’ll have a 95.64% chance to get our 5:1 ROAS target — well over 90%, making it admissible — even though every single one of those values is outside the range of the “local” guarantee for the relevant segment. With more admissible options to choose from, our optimization is free to pick one that offers better performance, under the same guarantee level.

Optimization

For any given segmentation, including the one where we pile all traffic into a single segment, optimization simply means finding the bids or max CPC settings from the admissible set that maximize the expected value of the campaign. The ROAS guarantee is a constraint, filtering out any option with over 10% chance to drop below our 5:1 ROAS target; we then just pick the best of the remaining options.

Obviously, slicing and dicing our campaign into different segmentations can make a huge difference. Let’s optimize our campaign for gross revenue:

With the entire combined traffic as a single segment, the optimal bid or max CPC is $0.74 — the highest admissible, no surprise there — which gets us $22,626 in expected gross revenue.
Optimizing our three segments drives the bid or max CPC setting of segment A up to $0.91 and lowers segments B and C to $0.67 and $0.58 respectively, pushing our expected gross revenue up to $25,577.

That’s over 13% more revenue! Crucially, both settings gave us the same 90% guarantee for a 5:1 ROAS or better, but segmentation allows us to bid much more efficiently on clicks of different value, for significantly better performance.

A “deficient” segmentation is double whammy inefficient: for some of the traffic we bid too high, performing worse than our ROAS target; for the rest we bid too low, losing out on traffic that would have met our performance goals, sending it over to the competition. Our ROAS guarantee holds anyway — some segments subsidize others, so overall we hit our target with that 90% chance — but good segmentation allows us to callibrate our bids much better, maximizing the total expected value of the campaign.

Segmentation plays an even bigger role when we optimize for “bottom line marketing contribution”, subtracting the expected cost of our campaign from its expected profits — welcome to the adult table. Suppose our 5:1 ROAS target is based on a 20% average margin rate, so really we’re guaranteeing a 90% chance the campaign will make a positive net profit:

Without segmentation, the optimal bid or max CPC setting is $0.61, generating an expected profit of $3,240 (= 20% of $16,202 gross revenue) at a cost of $2,507 — for a net contribution of $734 to our bottom line.
Segmented, the optimal bids or max CPC settings are $0.72 / $0.52 / $0.48, generating an expected profit of $3,915 (= 20% of $19,573 gross revenue) at a cost of $3,006 — adding $908 to our bottom line.

That’s over 23% improvement in the bottom line performance of our marketing, simply by using our data. Better yet, we could look at the actual profits of individual historic transactions, to infer margin rate distributions for each of our segments — different ads and keywords may drive sales of different products with different margins, geographic location may impact shipping costs, et cetera — and allow optimal segmentation to take all that into account. Let’s use all our data!

Optimal Segmentation

Shouldn’t we segment our campaign further then?

Maybe, since segmentation does come at a price. Roughly speaking, less data in each individual segment means more uncertainty regarding their performance, and more segments means more uncertainty regarding how traffic gets split between them. As we continue to segment our campaign into finer partitions, our ability to make ROAS guarantees diminishes, until at some point we’re constrained into worsening expected performance. A more complete explanation requires that we get into the math, but the intuition is that it’s a balancing act: further segmentation allows higher resolution optimization over a smaller admissible space, where the benefits from the increased efficiency may be outweighed by the restricted domain.

Ultimately, we’re looking for the optimal segmentation for our optimization target: in our examples so far, it’s either the one that gets us the highest expected gross revenue, or the one with the highest net contribution, once we optimize our bids or max CPC settings constrained to a 90% guaranteed 5:1 ROAS for the campaign overall. Naturally, different optimization targets may imply different optimal segmentations.

Actionale & Differentiating

There are many (many!) ways to segment a campaign, not all of them meaningful in our search.

We should only consider actionable segmentations, supported by the channel — if we can’t set a different bid or max CPC for different device types, geographic locations, or new versus repeat customers, then those are irrelevant for optimization in this channel. Bid adjustments, also called modifiers or multipliers, are just a roundabout way of altering bids in distinct segments (except some platforms limit how big the difference can get); negative keywords and other exclusions effectively set a zero bid for their target.

Next, only differentiating segments can increase the efficiency of our bids: those that have contrastive, dissimilar enough inferred distributions, so that a different bid or max CPC in each would give us better performance than a single common one. There’s little value in breaking up groups of keywords or ads with near identical behavior. Age and gender can have a strong influence over the value of a click if we’re selling apparel, maybe not as much for organic fertilizers or office supplies. We don’t need to guess: our inference gives us those insights directly.

Other Campaigns, Other KPIs

This general approach applies to any type of campaign, optimizing for any KPI.

If we’re paying for impressions, we’ll infer value per impression distributions, instead of per click — we add the math for modeling the click-through rate of the different segments since our funnel starts earlier now, and we may want to optimize frequency caps on top of the bids, but otherwise we follow the same logic. Whatever we’re paying for, we’ll infer the distributions of the value per that thing — transactions, leads, emails with varying discount coupons, …

Life-time value, subscription business models, seasonality, incrementality (fractional attribution, organic cannibalization) and other refinements involve changes to how we make our inferences. The math gets slightly more complicated, but the general principle remains unchanged: quntifying our uncertainty regarding the future value gives us strong performance guarantees, before the campaign even started, and the ability to optimize that performance to the fullest potential of our data.

K2DQ

Take advantage of K2DQ’s solutions for marketing teams and agencies. Let’s talk :)