TMW #178 | The dying art of AB testing

Jun 9, 2024

Welcome to The Martech Weekly, where every week I review some of the most interesting ideas, research, and latest news. I look to where the industry is going and what you should be paying attention to.

👋 Get TMW every Sunday

TMW is the fastest and easiest way to stay ahead of the Martech industry. Sign up to get TMW newsletters, along with a once-a-month complimentary Sunday Deep Dive. Learn more here.

The dying art of AB testing

A different kind of industrial revolution

When the industrial age came about, there were countless professions that slowly and then suddenly went bust. By the mid-19th century, people who hand-made chairs, shoes, and stagecoaches were all subsumed by steam-powered machines.

Coal Pits & Factories

Now in the Aification of everything, we’re seeing another kind of displacement. Not of physical, handmade craftsmanship, but of knowledge work. And one of the categories that could become a dying art is AB testing. 

The idea that AB testing is a dying art is not controversial; it’s like saying that horses are not good cars. But I get that some of you might be upset by the idea. Experimentation is a major driver of the marketing technology industry, spawning huge companies like Optimizely, SiteSpect, and Contentsquare. Years ago, Meta’s CEO Mark Zuckerburg argued that AB testing was the main reason behind the early success of Facebook. 

AB testing was also where I got my start – running AB tests for dermatologists and bike shops at a tiny CRO consultancy. It was fun, but grueling work. 

With the emergence of Generative AI, we’re seeing the death of AB testing as we know it. Or are we? Will AI impact AB testing at all, or are we seeing a rebirth of a category that’s vital to digital performance into something completely new? Let’s find out. 

🔓 Continue reading with TMW PRO

Get the entire Sunday Deep Dive and all of our newsletters every week with TMW PRO. It's the easiest and fastest way to make sure you're fully informed on how the marketing technology industry is evolving. 

Here’s what some of our existing TMW PRO members have to say;
“TWM is a finger on the pulse of what's happening, with the clarity of insight to see the important patterns underneath the noise” – Scott Brinker (VP, Hubspot).

“I recommend TMW to all leaders who are customer-focused and transforming companies via digital technologies” – Kazuki Ohta (CEO, Treasure Data).
“I've subscribed to many Marketing newsletters, but TMW is at the top of my list with the most valuable content for marketing professionals" – Lilly Lou (Marketing Director, Icertis).
TMW PRO is an invaluable resource that’ll help you to truly transform and future-proof your career in Martech. 

Are you ready to join the PRO's?

If the answer is yes, CLICK HERE.

Stay Curious,

Make sense of marketing technology.

Sign up now to get TMW delivered to your inbox every Sunday evening plus an invite to the slack community.

Want to share something interesting or be featured in The Martech Weekly? Drop me a line at

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.