MSoM #016 | Experimentation and the ROI myth

Nov 28, 2021

Listen on Apple, Spotify, Google and everywhere else.

Welcome to Making Sense of Martech, an irregular set of interviews with some of the most interesting people in marketing technology.

Making Sense of Martech is an extension of the TMW newsletter. Each week readers ask me to go deeper into the most popular topics of the newsletter. Making Sense of Martech helps to add additional context into what's featured every week by interviewing the people behind the research, ideas, and important work.


In this episode of Making Sense of Martech, I’m joined by two guests, both in their own right leaders in experimentation. First, we have David Mannheim. He's the Global VP of Conversion Rate Optimisation with Brainlabs UK, and the former founder of a leading optimization firm, User Conversion. Second, we have Oliver Palmer, he’s a senior optimisation consultant and prolific author who’s worked on a number of high-profile brands in the APAC region.

Across the digital experimentation industry, the past year has seen a major shift in how people think about the place of optimization in their businesses, questioning the validity of associating testing with revenue outcomes, and the scientific promise of experimentation. Several articles featured in TMW have been authored by our guests discussing and exploring this topic. From David, a series of articles exploring the connection between experimentation and financial gain and from Oliver, tracing the crisis of optimization and its detachment from its scientific methodologies. In our conversation, we talk about the changing perspectives on the purpose and practice of optimisation.

Listen on Apple, Spotify, Google, and everywhere else.

You can find Oliver Palmer on LinkedIn and Twitter and you can read his articles here.

You can find David Mannheim on LinkedIn and Twitter


📚 Resources

Oliver Palmer | The crisis of optimisation

Oliver Palmer | You probably don’t need A/B testing

David Mannheim | Why do we assume experimentation is about financial gain?

David Mannheim | No, you can’t accurately attribute, nor forecast, revenue to experimentation. Here’s why.

David Mannheim | Yes. We need to attribute revenue to experimentation. Here’s how: Part 1

David Mannheim | Yes. We need to attribute revenue to experimentation. Here’s how: Part 2

Y Combinator | Mark Zuckerberg : How to Build the Future

Duncan Wardle | Why I Left Disney

Juan Mendoza

Marketing technology strategist at The Lumery, I analyse marketing, data, and technology trends for some of the most well-known Australian and global brands.

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.