TMW #159 | Measurement and the tyranny of the abstract

Welcome to The Martech Weekly, where every week I review some of the most interesting ideas, research, and latest news. I look to where the industry is going and what you should be paying attention to.

👋 Get TMW every Sunday

TMW is the fastest and easiest way to stay ahead of the Martech industry. Sign up to get TMW newsletters, along with a once-a-month complimentary Sunday Deep Dive. Learn more here.

I read a fascinating essay this week that has nothing to do with marketing or tech. It’s actually about lifting weights. It’s not about becoming some crazy techno-loving, tank top-wearing meathead. 

The author, Jordan Castro, brings something into the essay that I did not expect – the idea that our bodies and minds are often conceived as two different things, and how that can be a big problem. He calls it the tyranny of the abstract: 

“The world is increasingly fractured and disembodied, traversable in its entirety from a living room, where one disjointedly imbibes disparate bits of information on a screen, in a way our ancestors wouldn’t have been able to conceptualize. Thoughts, discourse, and even self-conception are let loose in the realm of abstraction; “reality” is an imposition on our “freedom” to mentally meander—and so we do our best to blot it out. But one cannot blot it out. The tyranny of the abstract leads to acute and frenzied suffering.”

Extending this idea to the world of online digital measurement and all the analytics software packages, spreadsheets, reporting, and insight-finding that happens every day… something irked me about what Castro is saying here. And it’s this: our disembodied avatars traveling through social media apps, websites, and, of course, ads – mediated by LCD screens and smartphones – somehow don’t do justice to how we understand each other online.  

And this is no more important than in the domain of measurement; that is, the analytics practices that we’ve adopted in our businesses to help us understand our customers. But how much can you really learn about someone when we are massively impaired by the simple fact that the things we measure are completely cerebral and not embodied at all. 

Another way to describe this limitation is the always classic (and kinda funny) illustration of the parable of the six blind men and the elephant.

As you can see, it’s a parable about perception. Everyone in the cartoon thinks that they know what an elephant is, but because they can’t see the entire picture, they make grave errors that border on the nonsensical. 

The elephant is a warning about our limited perspective. But could it be more limiting than reducing people to numbers on a screen? 

In the world of business analytics, it goes something like this: When we’re analyzing a dashboard or a report, or perhaps pulling up a fresh SQL query, our perception of what’s really going on with our customers often has very little to do with the numbers, but everything to do with what our vantage point is. 

The CEO, the marketer, and the data analyst regularly have different opinions on the same data. That’s our perception at work. So in this picture, who’s correct? 

In this essay, I’m going to put forth an unpopular argument in the world of Martech. But it’s one worth making to foster an open, honest conversation in the industry, that’s if we’re actually interested in truth. 

This is the argument: A lot of our measurement methodologies, software, and digital technologies that deliver incredible precision in analyzing our digital performance and give us insight into our customers have fooled us into thinking that they could ever properly reflect reality. 

What do we lose when we abstract away the humans buying, using, browsing, and learning about our products and services? We lose everything we came to analytics for: Insight. 

Welcome to the tyranny of the abstract. 

Blood, flesh, traffic, clicks 

Digital marketers have conflated heatmaps, conversion reports, web traffic, and the almighty “click” as a central dimension of how we can observe and modify human behavior. It has led to enormous growth for the analytics industry, predicates the value of online advertising with marketers, gave us Google Analytics, and a multiple billion dollar Adtech and Martech industry.

It was only a few years ago, for instance, that Salesforce paid more than $15 billion for Tableau, a company entirely reliant on this idea that marketers want abstraction of digital so much that they are willing to hire analyst teams to interrogate the data to give companies a picture of reality. 

The first premise I want to bring to this essay is the idea that the only way to truly understand someone is in the flesh. That is to see them in person, their behaviors, their facial reactions, body language, maybe smells. So much is lost by reducing a person to quantitative data, a mere number on a screen. But companies are attempting to go beyond the digital to create opportunities to collect and analyze “embodied” data, reintroducing a qualitative layer to rehumanize reporting. 

Amazon’s Go stores, a novel concept of a convenience store where payment is automatically calculated via computer vision surveillance, is a great example of attempts from Big Tech to bridge the mind-body connection in analytics. 

The idea is that by identifying every single person who walks in and sticking hundreds of cameras on the roof, you can get a clearer view of how customers react to products, how they travel down the aisles, and the drivers behind their purchasing decisions. Instacart is pursuing a similar strategy with its new advertising features integrated into real shopping carts. 

You might think this is Amazon’s attempt to break into brick-and-mortar stores, but it really isn’t. It’s a thinly veiled attempt at solving the obvious analytical gaps for the world’s largest e-commerce platforms – observing real humans doing real stuff. 

Amazon is the gold standard of data-driven culture, that ruthlessly follows the integer where it leads them. So much so that the FTC is currently suing Amazon for its optimization of the Amazon Prime online upsell and retention tactics. 

I’m sure the product managers and UXers didn’t get to the current state of Prime without a long and involved AB testing program, where the answers for growth led down a path no one wants to go down; one where users are being deliberately misled. 

Amazon should be able to sell its products however it likes, but the argument goes that the concentrated focus on optimizing how users move through a purchase flow can lead to some pretty dark places. 

The only reason we have so many dark patterns on the web is because we learned about them through web analytics. If these tactics didn’t deliver the desired result, then the numerous and growing list of dark patterns wouldn’t be on this list. 

The contentification of everything is a concept I introduced last year as a way to describe the perverse incentives of online media as it races to the bottom of the brainstem seeking clicks, attention, and traffic. That race is still alive and well: I recently went on a clickbait hunt and came across this news site the other day, and it’s a great example of what happens when everything is ruthlessly optimized for clicks. Just look at these glorious headlines: 

Multiple studies have examined the prevalence of content farms, clickbait journalism, and the constant optimization of media for attention, and all the effects are overwhelmingly negative. It diminishes societal trust in online media, gives credence to misinformation, pollutes our knowledge economy, and sometimes leads to disaster for people who take the content to heart.

For news media, the internet has disfigured the profession beyond recognition. Yes, society’s late-stage capitalistic urge to eek every percentage point increase out of every crevice of the digital experience plays a huge part. But it’s really the online analytics platforms that have enabled these bad incentives.  

By thinking less about the real people who read their content and more about the incentives of SEO or virality on social media, news media has only dug themselves a hole that’s only getting deeper. What you measure matters, because it’s what you eventually believe to be important to your business. 

Mr. Beast is another example here. There is no other creator like him because of his total dedication to YouTube metrics and complete abandonment of creative integrity. He’s held up as a hero of content creators spawning countless lookalikes. He figured out early on that doing videos about money (or the promise of winning money) drives huge engagement on YouTube and inadvertently created a cult that worships money and wealthy people. 

So clearly companies, content creators, and media outlets are learning something from all this online data. But my argument here is that while it can play a role in growing companies, we lose a lot in our overreliance on what is a very dirty shade of perception. 

A dirty window 

So why do I think that web analytics represents a dirty window on the world? Well for the very reasons I mentioned above: in almost every case, web traffic reports, conversion analytics, and ROI assessments only tell us about what’s happened, and not necessarily why.

NPS surveys might shed some light on why customers do things, and perhaps a handful of customer interviews can certainly help, although watching hundreds of session recordings is a stupid but promising way to get to the why.

But fundamentally each person is unique in their behaviors, and their reasons for those behaviors are multivariate and hugely complicated. 

Analytics is only valuable in the aggregate. And therein lies the ultimate flaw in using analytics to perceive reality in any kind of meaningful sense to understand humans. To understand people, you need to spend time observing them. A task that is impossible at any kind of scale or aggregation. 

Of course, AI will play a role in aggregating the unaggregated, but in the end, you’re just relying on AI abstractions to give you a sense of truth about your customers. The risk is that marketers take the bait and prefer to use AI to do their job of data analysis and, dare I say, actually trying to understand people. 

Either way, AI’s role in filling the web with bots does raise suspicions about what we’re seeing in our dashboard reflecting real people. I would argue that about 40 – 50% of what you see in the data are not real people. It’s probably more, but I want to be optimistic. 

Besides this practical element, even if we could get a perfect knowledge of our customers from a data perspective, it’s a fruitless pursuit. 

Prominent 20th-century philosopher Ludwig Wittgenstein once argued that even if we had the words to fully convey reality, we would probably not be able to understand it. Because language derives from agreements of what words mean and a shared understanding between people. It’s entirely subjective. 

The very sense of truth and reality is conveyed through language, and language corrupts and distorts. Metrics and measurement are just a subset of language, and to get truth in the way I’m defining it here is utterly impossible. It’s a hopeless effort. 

The job of the data analyst is on one hand impossible, and on the other, entirely necessary. You need some semblance of truth, but what is really truth if it’s about as clear as a dirty, broken window? 

Therein lies the tyranny of the abstract: nothing is ever fully real; it’s always just a step away from the truth – and even worse, that reality is refactored through lenses of varying colors. But what are those colors? 

The layers of abstraction

To understand the tyranny of abstraction, we must get a sense of what those lenses are. I propose a few concepts. But first, I need to get something off my chest. 

People tend to look at data analytics as this thing that’s similar to Lego bricks. I have lost count of how many times I’ve seen this image shared online: 

It’s a deceptive image because, in the realm of online analytics, we’re not dealing with real physical objects. We’re dealing with abstract ideas and concepts that represent something real. It’s a false approximation.

I propose that instead of assembling something out of Lego, data flows through various layers of abstraction and arrives at the person who has the power to make a decision, or to whom being influenced by that data is important. 

This is what it looks like: 

The core reality is the thing that actually happens online. A customer clicks on an ad, buys something, watches a video, or responds negatively to a survey. Of course, in this example, it’s all online for the sake of argument. But what happens next? 

The next layer is the data collection piece. We’re already losing signal from the real thing that just happened, because analytics software is designed to collect, aggregate, define and display data in specific ways. How Google Analytics defines what it collects is very different from Adobe Analytics: the meaning and reflection of truth is already abstracted, before the analyst actually gets to analyzing. Which brings me to layer three. 

The analyst takes to analytics packages and starts to analyze data and draw conclusions from it, whether that be in Excel (which is still – bafflingly – the common way) or in the tool itself. 

There are presuppositions brought to the data, connections made, and comparisons drawn. But at some point, all data is worthless without some kind of presentation back to someone who makes decisions (even if that someone is an algorithm or AI system). Layer three is all about interpreting the data. 

Layer four is presenting it back, usually with visualizations, some key insights, recommendations, and sometimes with an actual story. By now, the original facts have gone through several modifications. By the time it hits the perception layer, where people are listening to your story and looking at visualizations, all we’re left with is a scant memory of the reality that the data, story work, and analysis are supposed to convey. 

You’re correct to think that most reports that you read have very little relationship to the reality they are trying to convey about your customers and business. Is it still helpful? Absolutely! But does it reflect a true understanding of your customers? Probably not. 

Mind over matter 

We’re going through a bit of a measurement reckoning in 2024. The decline of cookies and the embrace of a more private web is not just a technological shift. It’s also deeply challenged the direction of our own careers in analytics and online measurement, and the time we’ve spent on what could ultimately become a worthless accumulation of skills and experience.

That’s why understanding our philosophical and empirical underpinnings of analysis is even more important to probe. 

As I’ve established here from the standpoint of web analytics, we’re all looking through a dirty window trying to make out the shadows, moving shapes, and the fuzzy figures within it. But it’s the best we have in an imperfect tech industry. 

My only advice is to choose the paths that help you best understand humans, because analyzing clicks alone will eventually become a fast track to ruin. The tyranny of the abstract is the company that seeks to blindly follow the data wherever it leads. Be the company that seeks to truly understand people; not just what they click on. 

🔓 Are you ready to join the PRO's?

Get the entire Wednesday Martech Briefing and all of our newsletters every week with TMW PRO. It's the easiest and fastest way to make sure you're fully informed on how the marketing technology industry is evolving. 

Here’s what some of our existing TMW PRO members have to say;
 
“TWM is a finger on the pulse of what's happening, with the clarity of insight to see the important patterns underneath the noise” – Scott Brinker (VP, Hubspot).

“I recommend TMW to all leaders who are customer-focused and transforming companies via digital technologies” – Kazuki Ohta (CEO, Treasure Data).
 
“I've subscribed to many Marketing newsletters, but TMW is at the top of my list with the most valuable content for marketing professionals" – Lilly Lou (Marketing Director, Icertis).
 
TMW PRO is an invaluable resource that’ll help you to truly transform and future-proof your career in Martech. 

Are you ready to join the PRO's?

If the answer is yes, CLICK HERE.

Stay Curious,

Make sense of marketing technology.

TMW is the fastest and easiest way to stay ahead of the Martech industry. Sign up to get TMW newsletters, along with a once-a-month complimentary Sunday Deep Dive.