Is your digital marketing campaign working? Can you trust that your ROI is accurate? Are you overspending on paid media? Randomized Controlled Trials, RCT (sometimes called “Randomized Control Testing”), can help you answer these questions and more, even as third-party cookies disappear. Learn how!
Cookies are disappearing, privacy regulations are intensifying, and challenges to effective data collection are on the rise. Many advertisers worry that without individual-level tracking, gaps in their media attribution models will make it difficult to measure the true results of their marketing efforts or justify marketing decisions using ROI.
To maximize your marketing impact, we recommend that you meet all the new changes with scientific principles and statistical methods. Our Randomized Controlled Trial (RCT) approach helps your business build transparency, beat impersonalization, and mathematically identify and prove sweet-spots for your media investments in a cookie-less world.
What is a Randomized Controlled Trial (RCT)?
A Randomized Controlled Trial (RCT) is a type of scientific experiment that aims to reduce certain sources of bias when testing the effectiveness of new treatments. This is accomplished by randomly allocating subjects to two or more groups, treating them differently while controlling for potential noise, and then comparing them with respect to a measured response.
An RCT can also be used to optimize your paid media spend. With this approach we utilize the same scientific rigor to reduce bias, and we don’t rely on either out-of-the-box attribution solutions or cookies. Even as individual-level tracking disappears, our RCT designs allow you to prove out your ROI across media tactics (including non-digital media), channel, spend, or messaging.
How do Randomized Controlled Trials work for advertising?
A media RCT is an A/B test for paid and organic media. Since we can’t split media based on a list of individual customers, we split the media based on different geographies using robust, well-established methodologies.
This image is an example of a very simplified RCT test. As you can see, splitting a media audience into groups with basic parameters allows you to get a more accurate view of your paid media effectiveness and test different media efforts (e.g., tactics, channels, spend, messaging, etc.) to see their effectiveness. In this example, we eliminated paid media from geos E,F,G, and noticed that revenue decreased by $15,000 when we did. That means paid media is bringing in an extra $15,000 in revenue.
We also have options to run RCTs without going completely “dark.” The example in the image above shows that half of the geos got no paid search. That’s a non-starter for most folks. If you’re interested in this RCT thing but worried about the idea of removing ad spend for a “control,” read on to learn about methods we use to avoid having to “go dark” with our control.
What are the benefits of using a Randomized Controlled Trial?
Test new media strategies to understand their effects.
RCTs give you the ability to test new media tactics without human bias or the need for individual-level tracking. Suppose, for example, Facebook wanted to see what banner ads would result in more people registering to vote in the presidential election. If they decided to put ads up about being a Democrat or Republican they’d get skewed results because the driving factor to click is based on user beliefs.
To solve this, Facebook employs RCTs in their Conversion Lift program to assign random controlled groups by geographies and test ad effectiveness without introducing bias. The Facebook-touted benefits of that program are exactly the same as the benefit of our RCT designs:
"Measure real business value, not clicks. Prove return on ad spend (ROAS). Learn if your attribution model is accurate. See the causal impact of your marketing"
In fact, since its growth in the 2010s, Facebook has seen the value of randomized controlled trials to accurately measure their advertising program success, noting, “There are too many unknown variables to accurately measure media lifts using traditional measurement tools.” Indeed.
Determine what efforts are actually driving results.
An RCT can help determine the effectiveness of your media results. We break down your entire media strategy by channel, spend, messaging, etc., to determine what is working well with your customers and what is not. This will help you determine where to properly invest your media spend to make sure you are delivering on the results you are held to.
Explore ways to optimize and adjust media spend.
RCTs provide deeper insights about your paid media efforts. Once we understand the overall breakdown of your media efforts, we can then suggest ways to improve them. We back that up with data and evidence to support how much you should spend, what messaging resonates with your audience, and what channels are the most effective.
Create measurable media results.
RCTs help create measurable media results. When you launch an RCT to understand what truly is bringing in the right customers and the right revenue, you can then set KPIs for your media efforts that are tangible and backed up by data.
How should I use an RCT?
Multi-Touch Attribution (MTA)—the attempt to accurately assign value to each channel or tactic that customers or prospects may encounter during their paths to purchase—is the way most businesses (or agencies) measure the success of their media efforts. But in a Kantar pole of 468 senior marketing leaders worldwide, 54% of respondents identified MTA as one of the “biggest gaps” in their marketing research.
Traditionally, media attribution models, including multi-touch attribution, promise a lot, but the results, unfortunately, can be wildly inaccurate. MTA also requires individual-level tracking across multiple touchpoints—a methodology that’s crumbling along with third-party cookies.
As available user-tracking data diminishes, we recommend an approach to optimizing media spend that combines media mix modeling and controlled experimentation.
The media mix modeling is the “quick start”—results will vary based on the availability of the data, but the real value kicks in with controlled experimentation, which starts with answering the highest priority questions about the impact of media investments. RCT is the gold standard method to capture the causal impact ads have on a brand’s business, and, once established, it should be employed as part of an ongoing process of experimentation to optimize media.
How can Randomized Controlled Trials be customized for my advertising program’s needs?
Each RCT that we design uses proven methodologies but can be customized. Our approach for separating “test” from “control” audience is determined based on the needs and constraints of each experiment. Here are three design options:
- Basic Blocking- We identify geographic areas that can be marketed to and that results can be measured against based on demographic and purchasing/listening/viewing similarities as available and run media “on” and “off” for like groups.
- Factorial/Multi-Stage- The same approach as basic blocking, but then different blocks are exposed to different combinations of marketing tactics over time to achieve similar results from running “on/off” experiments with a single experiment.
- Stepped Wedge- The same approach as basic blocking, but exposes all groups to some media eventually, but not concurrently. These designs work well when we need to treat every unit, but we will still want a causally-valid control group.
What are some use cases for using RCT?
To report the impact of paid media that can’t be digitally tracked (TV, radio, OOH, wallboards)
Use an RCT to determine the effectiveness of your media strategies and get a better grasp of what is working and what isn’t, beyond a simple dashboard.
To prove to stakeholders where to invest more money (or stop investing)
Use an RCT to measure and optimize media spend ROI and use as a case study when asking for more investment.
To understand where paid media is most effective (and adjust accordingly)
Better understand where your paid media is working and adjust your efforts with our guidance and suggestions based on RCT results.
When paid media results exist, but they’re not transparent
An RCT can create a better view of which strategies (e.g., spend, channels, messaging, etc.) work with your audience and which don’t.
To test a new tactic or market before investing a ton of money
Create a small RCT to test if a new market or channel will yield a strong ROI before investing a ton of money.
When your paid media budget has been cut, but you’re still held to the same results
An RCT can help determine where to invest to meet expectations and results.
What kind of results can I expect to see from using an RCT?
The primary result of using an RCT will be your ability to measure the cause & effect impact of your ad spend. This is huge for your business. You’ll eliminate overspending on media that may not be working for you, reducing your financial risk by 90%-95%. You’ll finally be able to one-up wise, old Mr. Wannamaker (1838-1922).
Secondarily, you’ll get an accurate count of your conversion rates, which you’ll then be able to increase with the knowledge you’ll gain. Facebook estimates that 37% of conversions are missed using cookie-based data. Without RCTs, these rates will only deteriorate as cookies continue to crumble.
Here are a few results RCTs have brought our own clients:
- 20% Conversion Rate Increase for Leading Healthcare Provider in Three Months
- 67% Conversion Rate Increase MoM for High Tech Company
-$10M Optimized Quarterly Media Spend for Global Pharmaceutical Brand
How do I know if Randomized Controlled Trials are right for my needs?
Short answer: they are. But if you’re still not sure, reach out. If you’ve read this far, there’s a good chance this is the solution you’re looking for. Fill out the form below and we’ll help connect you with the right solution for your particular challenges.
We can send more detailed information, including case studies wherein our RCT designs helped our clients see a 40% increase in qualified leads, reduce media spend while creating better results, and enter new markets with confidence.
