TMW #011 | New science on AB testing, Neo bank design principles, Etihad and digital transformation

Welcome to The Martech Weekly, where every week I review some of the most interesting ideas, research, and latest news. I try to look to where the industry is going and make sense of it all.

Here's everything you've missed in marketing and technology this week 👇

🎉 Just so you know  

This week I've launched email subscriptions and a slack community for The Martech Weekly! Now that subscriptions are available I will be transitioning to two versions of TMW - a full version that you can get by signing up for free and a public version which will be about 50% of the subscriber version. TMW #011 will be one of the last full versions for public reading. Go here to sign up

👌Top #3

🎭 New ways to measure AB tests. When it comes to running online experiments there's a lot of diversity of thought on how to measure the results scientifically. Frequentest or Bayesian there's a few different ways in which controlled experiments are determined as successful and we're years away from companies achieving the kind of digital maturity to take notice of how these models influence business decisions. The usual way for selecting a winning test is what's called the Null Hypothesis Significance Test (NHST). This is a methodology to determine whether or not we can reject the control (default) experience over the variation experience. With NHST there can be a lot of opportunities to introduce bias into the decision making process (like pausing or stopping a test) or making a decision based on confidence intervals, knowing that a confidence rating is just the likelihood that the change is causative to results, not that it is. An alternative approach which is growing quickly is sequential testing. Sequential testing seeks to select the highest performing variant in a group set on thresholds for what determines highest performing tests. There's always going to be a debate around how to measure AB tests, but it's great to see some fresh thinking around this topic, offering alternatives to the null hypothesis model. Link

💸  How Neo banks are disrupting finance through experience design. A great overview of the design process of one of Australia's fastest growing banks - Up Bank, came out this week highlighting the meticulous process in which their financial products are designed for user centricity. The fintech, banking, mortgage lending and insurance industry is currently undergoing a process of decentralisation - there are more banks, lenders and insurers that are pure play digital than ever before. So how are banks like Up competing? Aside from offering low cost financial products the big plays here are on experience, shifting best practices from digital experience leaders like Uber and Instagram into their own methodologies. For Up,  experiences are designed around reducing friction within the process but also adding important emotional moments within touch points, like a sign up flow completion. A great little insight here is that Up works hard to make even giving feedback on the product experience extremely easy and simple, giving an upside to customers and powering their iterations with rich data. Link

🛫 How Etihad is transforming their digital offerings during the  pandemic. Here's a great behind the scenes look at how Etihad airlines are transforming their business during the pandemic through focused, outcome led business discipline. Like almost every airline around the world, Etihad has seen a fair bit of disruption to their customer's behaviours which conversely has accelerated innovation in many cases rather than stifle it. The key aspect to this kind of transformation is business alignment and focus on what is going to drive tangible business outcomes in an environment where "out of scope" are luxuries. Etihad are doing this through focusing in on their customer's fears, friction points and concerns during COVID-19 in a way that allows for business intelligence to be actioned within their digital teams. They have spun up a few interesting products such as a COVID-19 self assessment tool and interactive travel maps and making it easier for customers to talk to a real, living and breathing human. For airlines that double down into the customer experience during this pandemic, there will be valuable digital assets on the other side, which will accelerate innovation even further when we all can travel again. Link  

📈Chart Of The Week

The 2020 Hotjar web analytics report in conjunction with CXL has been released this week, highlighting some very interesting trends around company size and use of behavioural analytics and voice of customer. The chart below speaks to the use of website feedback tools like polls and surveys to gauge customer sentiment. A highlight is the correlation between larger company sizes and their use of voice of customer analytics tools. This does not necessarily speak to the growth of such companies by using these sources of insight, but rather how enterprise businesses go about using different types of customer data, which tend to lean more to the qualitative scope than smaller companies. This reflects a broader move into voice of customer business intelligence which means that more enterprise companies are investing in UX and product design research methodologies to give needed context to quantitative analysis. Link

📚 Everything Else

How to build scale-able data environments for growing companies with Snowplow. Link

Accenture's take on the importance of first party cookies. Link

Six considerations around how Episerver's acquisition of Optimizely is one of the most strategic SaaS acquisition moves in recent years. Link

B2C brands are increasingly jumping into the advertising business. Sometimes if you want it done right, you'll have to do it yourself. Link

Meltwater and Cison and merging, which is going to increase the footprint of both platforms. Link

Kmart is going the way of IKEA with a bespoke AR experience with product recommendations based on customer histories. Link

How to build a dashboard to measure digital advertising effectiveness. Link

A great piece on how digital marketing should not be the quick fix companies are looking for during the pandemic. Link

Webflow, the popular no-code website builder has just ticked over 100,000 customers. Here's how the company was started. Link

Twilio's digital transformation road map was accelerated by 6 years during COVID-19. Link

Tiktok and its use of Australian user data. Link

A data security CEO was arrested on fraud charges. Irony at its best (and worst). Link

Stay Curious,

Make sense of marketing technology.

Sign up now to get the full version of TMW delivered to your inbox every Friday plus an invite to the slack community.

Want to share something interesting or be featured in The Martech Weekly? Drop me a line at juan@themartechweekly.com.