Skip to main content
Loading Explore guides…
Attribution in Marketing · Acedra
Home
Explore
Business
Attribution in Marketing
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
103 segments
Copy
Hide video
0:00
Definitely, what is universally accepted is that the third party cookies aren't going to be as powerful as they were earlier, and that attribution will have to be supplemented by other methods. So this trifecta of MMM, MTA and experiments being used to drive the right set of decisions for the right marketing brands is what all brands need to do, and they can pick the right methods and right metrics.
0:35
Hello and welcome to the AI Master Group podcast, featuring the people who are building the future of AI in their work today. I'm your host, Jim Griffin.
0:51
Today I am very happy to have as a guest Tapan Khopkar, who's the Executive Director, Marketing Sciences at OMD, which is the world's largest media agency network. He's also a member of the Advertising Research Foundation. Prior to that, he was the VP and Head of Innovation Labs at Cartesian Consulting. Tapan has his PhD from the University of Michigan, where he specialized in reputation systems. Welcome Tapan!
1:29
Thank you, Jim. It's great to be here, talking with you about topics which are of mutual interest. Mutual interest is exactly right. I've enjoyed so much working with you over the years. I first met you when you were the head of Innovation Labs at Cartesian Consulting. And you know, Tapan, that was really a great concept. Can you describe a bit about your role there, the kinds of work you were involved in, and the journey from there to what you're doing today? Sure Jim. So I think I'm going back
2:07
about 7 or 8 years. And I was working as head of Innovation Labs at Cartesian from 2015 to about 2020, and Innovation Labs was rightly said, and a really interesting idea. It was like the special ops within the Cartesian larger team, which was looking at customer analytics for the set of clients. But Innovation Labs was the team that would be involved when it is time to go to uncharted territories, be it new domains, new problems, new techniques. So that was the set of things that we looked at.
2:50
And like I was working with really great data scientists at that time. And while the work differed in, in different contexts, two key themes that, emerged out of that are still a large part of my work today. So one of the themes was around measurement. And if we are looking at measurement and even within innovation labs, we had built solutions to do, Marketing Mix Modeling and Multi-Touch Attribution. And I think we have collaborated on MMM for a few of the clients in the US across domains,
3:32
so one: We got a chance to do MMM across domains, across geographies, across KPIs. Or what is it that media is trying to drive. And that continues to be the key focus of my work at OMD today.
3:50
Now, MMM is aided by Multi-Touch Attribution and experiments, which forms the trifecta of modern measurement. So that is something that when we were doing those MMM projects, how to link MTA results to MMM is something that we were, like the rest of the industry, grappling with at that time. But now I think it is quite clear how these different solutions interact with and empower each other and help the marketer take the right decisions backed by the right measurement. So that's one theme
4:28
that, had emerged from my work at Innovation Labs, which is the backbone of my work at OMD EMEA. The other theme is probably more of interest to the AI Master audience, which is the use of LLMs and AI techniques and also behavioral economics, which is part of my background. So using principles from behavioral economics powered by AI to drive customers and targets, to nudge them to take the decision that brands want them to take. So that's something that we had pioneered. Kyte and SOLUS when I
5:07
was working with Innovation Labs. Now those solutions have
5:12
just been turbocharged because the principle, the behavioral economics principles are actually tried and tested. And those remain things like, fear of missing out. That's something that if you introduce a FOMO in a subject line, or if you introduce FOMO in a creative, then that is going to make people want to take the action that you want them to take. But how do you bring FOMO to life? Now the new techniques are much more efficient and much more nimble in doing that
5:46
in a variety of ways which are tuned to the context of the individual user. So the use of AI techniques and insights from behavioral economics to nudge the customers to take the decisions that brands want them to take, was the second theme back in Innovation Labs, and that continues to be an area of growing interest within the larger advertising community, and also an area that I'm looking at actively in OMD.
6:20
That's very interesting. Well you talked a bit about the backbone of your work and you just mentioned just know what you're doing currently. Can you describe your primary area of focus right now at OMD? Sure. So, I am part of the marketing sciences team at OMD, and we look at a set of markets within Europe, Middle East, Africa, Asia, but we also serve clients from the United States and Canada. So what we are doing is . . . There are two key pillars of what marketing sciences does.
6:58
One is measurement and the other is optimization. And we are a small team with a large footprint, is what I keep saying in all leadership meetings is because even if we are not directly involved in a client's work, the frameworks that we build for measurement are utilized because the media agency’s job is to optimally help the client drive their media objectives
7:25
(their marketing objectives actually) powered by paid media. That's what the media agency is doing. And for the media agency to do it optimally and to drive the right objectives that the brand wants to drive, marketing science plays a key role
7:42
in answering the following questions: like, What should be the right KPIs that the brand should focus at? Should they focus at the top of the funnel? Drive awareness? Should they focus and drive consideration? Or should they just be focused on conversion? And there are no right answers to any of these questions. You know, one-size-fits-all manner. It really depends on what the brand is and has been doing, what the brand intends to do. How is the ecosystem, how is the competition playing up?
8:15
All of these factors need to be taken into consideration to give a brand the right advice on how much media, support is required. What's the right way to deploy it? Measurement comes into play here because, What is the impact driven by media?
8:35
So if a brand is . . . Let’s say if I'm looking at a travel brand and if they are getting 1 million customers a month now out of them, a large number would have . . . If it's a reputed brand, a large number would have come to that brand, even if they were not doing paid media in a given week or in a given, month. But what is that number? What is the incremental impact of paid media over and above what the brand would have naturally got, based on it's
9:09
brand awareness, which is again built on historical media, in addition to the core experience of the brand. So this is called baseline, which is what is not driven by media. Then over and above that there is promotions. There is factors like seasonality, like if it's a travel brand, then holidays play a big role in that, school holidays and bank holidays, all of these factors impact how people . . . The natural inclination of customers and prospects to want to come to
9:46
this brand is bolstered by media. So, but what is the incremental impact of media? To estimate that in a robust and actionable way is one of the key deliverables that my team provides for our set of clients and now this can come in a variety of flavors. So at a really abstract and high level, strategic level there is setting up the measurement practice for a brand. What's the right way for . . . like, What should be the KPIs? What should be the factors that they consider
10:24
If I want to estimate the impact of media, then the impact of media has to be estimated accounting for other factors that are driving the sales. Now this is something that we call drivers analysis. So drivers analysis will tell you, what KPIs should you look at measuring and what are all the factors that you need to measure if you want to do an accurate estimate of the impact of media. So, like making a recommendation of, How should your measurement framework look? Auditing people's
11:03
existing measurement systems and measurement frameworks: That's at a conceptual level. Then at a more operational level there is running MMMs, doing fresh MMMs or refreshing old MMMs that are done, either by our team in the previous years or by somebody else, and making those more robust, making them more actionable: Is the measurement pillar. The other pillar is optimization. Now, in the optimization pillar there are a couple of standard offerings. One is, what's the right media mix?
11:43
And optimization is generally driven by measurement because once you do an MMM or once you have results of an Multi-Touch Attribution or if you have results of an experiment, then those results give you certain inferences. Those inferences need to be converted into recommendations. and then how to implement those recommendations to meet the brand's objectives in the upcoming period, given the context, the business context within which the brand is playing, that's where optimization,
12:16
deliverables, come into play and budget setting or budget allocation across product line or across market, that's one standard deliverable. Then the other thing that we do is: What's the optimal channel mix? Now, once you do an MMM, then you will know that, say for the said travel brand that we were looking at, if 10% of the sales are driven by media: Now, out of those 10%, maybe half of them were driven by TV, 30% were driven by social media, and 20% were driven by search,
12:57
just as an illustration. If this is the way it was done, then if TV was driving half of the sales, and yet if TV was allocated 80% of the budget, then there is a course correction to be taken. Now, this is . . . Like it may not always play out this way, and driving short-term sales is not the only thing that TV is doing, so I'm not going to recommend that, shift 30% of the spends from TV to other channels. No. A change recommendation is usually a gradual one and it and MMM naturally dovetail
13:37
into setting up of experiments. So once you get results and inferences from an MMM, they point in a certain direction. They say that correction needs to be made in this direction. And unless it is a drastic situation, one would not make a drastic measure. One would make incremental changes
14:02
in the direction pointed by the model, but one would also try to bolster the results of the model, because a model is, after all, a correlation and it is a story that we are using to the data to tell ourselves and client a story that makes sense. But after all it is the correlation. Like it is a story based on a robust set of correlations, and it has to be bolstered by looking at the other elements of the trifecta, which is your Multi-Touch Attribution models, which continue to have some value
14:37
despite the deprecation of cookies, and experiments that you conduct. These could be lift studies powered by platforms like Meta or Google, or these could be geo experiments that you yourself commission. Not all experiments are easy to conduct. But if you have these three levers in place, then they can talk to each other and you can get more robust, actionable, modular insights for different set of decision makers.
15:11
So this, the optimization pillar, actually looks at this. And then there is other consulting projects like a financial services brand wanted to launch a new product in a few markets. And they wanted us to recommend what's the right way to launch that product? How much money should they be spending? How should they do . . . Like how should that money be allocated? Or what should they look at branding first and performance later? or performance first and branding later?
15:47
Or should they spend equally on both? And also through the year, how should they flight that? Should they do a burst and then some trickle? Or should they have an-always on kind of a campaign? So all of these recommendations based on the data that they had of their own historic product and of the competition. So that's a very interesting consulting project
16:14
that we worked with.
16:16
So that's the like. Again, this is more in the optimization space because measurement is a given for this kind of a project. But what do we do? And now I'm coming back to what I earlier talked about AI, the use of AI. So in like some of the offerings that we are looking at or also trying to mine behavioral data from the web, like if you're looking at social listening, then what topics are being talked about in social listening? The brand is taking efforts to drive certain conversations.
16:52
But is that really happening? Because what happens, Jim, is that measurement of brand, done by traditional research methods, is time consuming and expensive. So, what we have tried to do is develop something called brand heat, which is a behavioral accompaniment to standard brand measurement. Brand heat looks at behavioral data across search, across social listening, and all of these are processed using AI tools. And then statistical analysis is done to figure out what are the factors that are
17:38
providing your brand heat today, which will drive brand metrics down the road. So this is one way in which AI is becoming relevant and there is also stuff around creative and content optimization, which is very similar to the Kyte product that we had worked on earlier, which is a subject-line, email. . . So Kyte was an email subject line optimization product. Now with the advent, with the advances in LLMs, now that can be . . . The same principles can be applied to
18:15
not only text but also to images. So images which have a dog in them, for family oriented brands, images which have a pet in them are going to get to 20% more visits than images without a pet, all else head equal. Now where AI techniques are coming in is to process those images to detect the objects which are meaningful, where behavioral sciences are coming in is to set up the AI system in the right way so that you are getting the right kind of cues, the right kind of features can be extracted,
18:57
which will be useful for nudging customers to drive the behavior that you want. It's really a broad spectrum of work across the two pillars of measurement and optimization and statistical techniques, statistical models help us tell the story of what is happening. And AI techniques, the AI tools, help us figure out what is the right nudge to be driven to drive behavior of the customers.
19:32
Very interesting. I especially liked the example you gave of the financial services brand that's launching a new product. And then they have a number of questions, not only about channel, but also about flighting, and whether to do it in a burst, or burst and trickle, or Always On. those are very realistic questions that somebody that's running that line of business is really going to need to lean in on. I can really imagine those conversations. I agree with you, that's very interesting.
20:00
Another thing that you mentioned just now was some of the testing that can be done by Meta and Google. I'm aware that Meta and Google have released some pretty accessible tools of late. For example, Robyn. what's your opinion about the quality of those new tools.
20:23
Okay. So I'm going to give what might seem like a non-answer. So Google has released . . . It used to be called LightweightMMM and now they have rebranded as Meridian. Meridian is a Bayesian approach to doing MMM. Robyn is a frequentist approach and they use machine learning and time-series techniques to make it really easy, accessible, yet robust. So both the tools are fantastic. But a tool is only as good as the decisions, the quality of the decisions, that it drives.
21:01
Now, with the proliferation of MMM tools and MMM becoming more mainstream, the tools . . . Like we have our own, in-house tool within OMD that we use for our MMM projects. Now, all of these tools are actually really good now, like tools for MMM is becoming like infrastructure within the industry. The quality of the model is driven by the maturity of the measurement setup that you have, the maturity of the team that is doing the modeling. And most importantly, it has to drive decisions.
21:44
It has to drive action. So if the MMM models are not utilized, if they are not actionable, then they are going to . . . Even the best model is going to be on a printout or in a PowerPoint deck in somebody's Inbox, and it's going to be looked at the next time that
22:07
this activity needs to be done, maybe three years later.
22:11
It will be a great model, but with wasted potential. What is required is an ongoing measurement and optimization program, which is looking at,
22:29
incrementally conquering this land of uncertainty, of: What is the impact of media towards me meeting my business objective? is the question, but that is such a big question that it can't be answered at once.
22:52
You're listening to the AI Master Group podcast.
23:06
What is the impact of media towards me meeting my business objective? is the question, but that is such a big question that it can’t be answered at once. It can only be answered incrementally. So maybe in the first run you will understand that media is only . . . (paid media) Like if I'm an established brand that has been in the business for a long time, then paid media will drive maybe 5%, maybe 10%, maybe even 2 or 3% of your overall sales.
23:40
Of the brand perceptions, it will drive a much larger component. But, What is the extent of my business KPIs driven by paid media? is the first level of question. Then how do different media channels do it different? Like what is the relative power of, efficiency and effectiveness of the different media channels of the different media initiatives in driving that impact? Is the next layer, then going down to a creative level or going down to a campaign level,
24:15
Whether it's . . .Whether I should . . . So that's the even the next layer. So you can't, you can't achieve really nuanced decisions or inferences at the first go. You have to have a stable foundation. You can maybe answer a few questions. You can try to answer a few questions with a high degree of confidence and actionability, or you can try to answer everything at a go and be left with no firm decision, no firm insights. So setting the measurement right is really crucial.
24:57
And the quality of tools is not going to, be the differentiating factor in the quality of MMM models. The team, which is the business team, which is engaging with the measurement team, also has to be mature. They have to understand the
25:17
different roles that media is playing for the business, and also the power and the limitations of various tools. Now, these are really, I mean, some of them are technically really sophisticated models and solutions. So the discussions do need to be technical. But it should not end there. It will always be driven towards What is the decision or set of decisions that the model is empowering? That should be the focus. And if one is setting up measurement right, by taking this into account,
26:00
then you will have a robust, evolving, ongoing measurement and optimization program. Doing it once is great, but it's not enough. Yeah. You were talking about it being actionable and being ongoing. I remember a project that you and I got pulled into with a major CPG brand that was getting a quarterly MMM readout, but it was always getting it at the end of the quarter, and they had already set their budget and made all of their decisions by the time they got the report,
26:34
so it was more or less a history lesson for them. And the reason that they were bringing us in was to try to see if they could accelerate the insights so that it could be actionable. I remember that project.
26:48
No, I mean, it's still the same, like that remains a reality even to the day for a large number of brands, like once a quarter is really the best. Like you don't need to do MMM more than once a quarter, but the decisions, like it is horses for courses. So the quality of the . . . Depending on the granularity of the decision, you need to employ the right set of data and tools to empower that decision. But having a unified overarching framework, powered by MMM,
27:25
is essential for a large number of brands which have a significant media spend. If you are not spending . . . Like not everybody needs to do MMM. Now, what has happened: I don't know if this is where you are going, but with MMM becoming mainstream because, Google and Meta and TikTok keep talking about it,
27:49
many brands feel that they need to do this, but they don't really know why they're doing it. So what may be that for certain brands which are digital only, maybe attribution solutions, which are provided by the likes of Google and Meta, maybe they're sufficient. It really depends on what decisions you are trying to drive. Like I'm . . . I’m not advocating that people don't do MMM. But not everybody needs to do MMM all the time, depending on how much like, what's the . . .
28:23
How much media spend do you have? What is the variety in your media spend in terms of channels, across channels spends like the business context and the data quality will tell you what is the right set of tools for you to employ in a given market. It's also possible to use insights from one market or from one model
28:50
to drive decisions in other markets or other models. So, like having an actionable, a future proof, modular measurement setup is essential for everyone. MMM is for the evolved brands who are spending a lot of money. And MMM will be the core, a core player in that setup. But not everybody needs to do MMM all the time. One of the things that you talked about just now is there are cases where powerful tools, robust tools, fall into the hands of people who don't necessarily understand
29:33
all the implications of those tools. Maybe that you've got a couple of variables that have got multicollinearity, and then you throw them all in there, or you've got some hyperparameters that you really don't understand what they do, and you just set the defaults or, there are things that happen there. But there's another problem that I've heard you talk about as well. I've heard you describe people who torture a model to extract the answer that they want. Can you describe what you
30:01
mean when you say that?
30:04
Yes.
30:09
Measurement needs to be approached with an open mind. If you already know what answer you want from the model and what decision you're going to take. If you're like . . . This may be controversial, but if you're doing an MMM for somebody else, you might as well not do it. You need to be open minded in doing them, because what will happen is that Yes, you can tweak the parameters. You can . . . There are several nuances in MMM. So there is what is called as AdStock. So AdStock is . . .
30:46
So if I do a television advertisement today, then if I am that the travel agency, the travel portal that we are talking about. if I am the travel portal that we are talking about, if I do a media, if I do a television advertisement today, not everybody is immediately going to
31:04
do a booking because of my ad, but it will have an impact. It will continue to have a lingering impact. So this is often represented by certain geometric transformations. So if you did 100 GRPs today, and if your rate of decay was 80%, then it's also as if you have done 80 GRPs tomorrow and 64 the day after, and so on. So in an MMM model, you not only . . . You have a time series of your media, media activity in the MMM model, but you have an AdStocked set of terms coming in as well.
31:43
On the top of that, there are non-linearities. So if you look at the relationship between the media activity on the x-axis and the impact that it is trying to drive, say, site visits to the said travel portal’s website. If you look at that graph, then it's not going to . . . It’s very unlikely that it's a straight line. It will typically have a C shape or an S shape.
32:12
Now this is where between C and S, that's a matter of great debate and controversy. There are some people who say that it's always going to be an S shape. Where you see a C is only because you don't have enough data, but in theory, it should always be an S shape is what many people say. But what you often . . . because you have to have certain threshold below which media is not going to have an impact. So the S has a lower arm. It has a linear or up-to linear, like sublinear, arm.
32:42
And then it has saturation. So there are people who say that it's always going to be S. If you are seeing a C-shaped effect, it’s only because you don't have enough data.
32:55
But that is a very interesting discussion that I'm not going to get into the depth of, because that's a rabbit hole. But no matter what you do, Coming back to torturing, so you can tweak the parameters
33:13
in ways that . . . An expert can tweak a model and get you the answer that you want, but that's not going to . . .
33:26
If it's not really your true belief, you shouldn't do that. You shouldn't accept that answer because reality will find a way out to bite you back. Because you are going to take a decision based on that model. Now, if I say that, like TV is driving 80% of the impact . . . Sorry, even at a more fundamental level, if media was only driving 10% of the impact. But if I am able to somehow create a model where media is having 20% of the impact, now that means like . . . If I give you this result,
34:06
then that means a much larger media investment. But if media was truly not driving 20%, it's going to come and bite you back.
34:16
Now this is a mistake that can be made either wittingly or unwittingly. It's a dangerous mistake, which is why you should try to . . . Like when I'm building a model, I am trying to knock down every narrative that I'm coming up with. I'm really cynical, at my cynical, like most cynical. Like it comes quite naturally to people like us. But it's one has to be even more cynical when building a model. Whatever withstands the most cynical scrutiny is worth testing.
34:54
And, you make incremental, marginal decisions powered by the direction that the model is pointing you towards, but always look for other pieces of evidence which will back you up. So the Bayesian frameworks like Google recommends, Bayesian frameworks.
35:15
There are no universal rules, but where possible, we recommend Bayesian frameworks where it is. There are places where we use frequentist frameworks. It really depends on what is the business context that the client has.
35:30
But you need to get signals from multiple methods and try to bolster the confidence that you have in the insights that the model is giving you or reject the insights. Like it's good to set up an . . . If you are likely to be wrong, it’s good to be wrong in one market. And by a small amount, then be wrong in all markets by a very large amount. So you are likely to be wrong . . . If you torture a model to extract the information, reality will bite you back.
36:03
So, you can do it, but don't, and always take incremental marginal decisions in the direction pointed out by the model, is what I recommend. The other thing that is worth remembering is that media works on the margins. Media amplifies what is already out there,
36:26
but media cannot . . . Like media can't give you all the business. You have to have things. Like if you were an FMCG, you have to have distribution, you have to have like . . . If one of the alcohol brands that we work with. So there is on-trade and off-trade, which is within the restaurants, what you do, and off-trade is in the retail. So you have to have presence in all the restaurants that you want to be present in. You have to also have things like experientials.
36:58
So you have to have word of mouth. You have to drive trials. All of these things need to be there. Media works on the margin and amplifies whatever else that is there. But what competition does also matters. Also seasonality matters. All of these things matter. Media works on the margin is often a humbling reminder, but it's one which is essential, and we tell our clients that Yeah, I mean, sometimes it is like: Yes, we did really well, but media only had a marginal role to play in it.
37:34
It is an important role, but you have to take care of your distribution. Don't throw in money into this market for this brand unless you fix distribution, which is only at 17%. So if you don't have distribution, media isn't going to give you sales. So the job of marketing of a marketing scientist, is to help the media planners and strategists
38:07
make the right recommendations, not only for media, but also in terms of the business context. Now, if we don't have data, we will say that these are things that we need to look at because these are factors that might also affect the sales. I don't have distribution data for a . . . If I'm making a budget recommendation and I don't have distribution data, then I will say that these are the caveats that I have. I'm assuming that distribution stays as is.
38:36
If that is the case, then media will give you this lift. But, it is . . . Like the two three things that I try to remind myself and often in conversations we end up telling the client is that media works on the margins. Then, it is best to be skeptical. It is essential to have a model, but it's also best to be skeptical of a model. And always make incremental decisions and use testing. And make the decisions in the direction pointed by the model. But don't go all in.
39:14
You're talking about these incremental decisions and the testing and so forth based upon model and we've talked in the course of the discussion about Marketing Mix Modeling, MMM, which is a model. And you've also touched on MTA, Multi-Touch Attribution. And you've mentioned that each has its own role, and it's a little bit more sorted now. But can you tell me if these two methods of ascertaining the truth give different answers? How do you deal with that?
39:52
So that's a great question. And I think the . . . and this is something that we had been grappling with about 5 or 7 years ago, now the way that now . . . I think this is the established understanding in the industry now, is that these techniques are essentially trying to . . . Use these . . . Okay, I'm going back to horses for courses. So MTA will give you actionable decisions in a shorter timeframe at a granularity that MMM usually will not approach.
40:25
So use MMM for doing strategic decision making. And what we often do is that we put these two in lockstep. We look at the . . . For relative . . . Within a channel, across strategy A or Strategy B, across Creative A or Creative B, the MTA decisions are good enough. Don't use the MTA decisions for estimating the overall impact of media. That's not a . . . That's not a question that MTA is going to help you answer.
41:01
Within a platform, what works better than others is something that MTA will tell you. Now again there are certain platforms and certain channels where the MTA often overestimates. There are places where it underestimates. One can apply a correction factor or a calibration factor based on MMM, so that's something that a number of people have done: Meta studies on MTA results and on MMM results, and they have . . . It it's fairly sorted that for tactical, short-term decision,
41:41
granular decision making, MTA can be used.
41:46
Don't use MTA for strategic, long-term decision making. Yeah, that makes sense. So you're talking about this tactical versus strategic aspect. And the question that I have for you is: You're doing an experiment in a world where you have contributions from things that happened in the past, like you mentioned just now, about AdStocks and so forth, so those things have a lingering effect on the experiment. How do you handle that?
42:20
Great question.
42:25
When you are setting up an experiment, I think you handle that in two ways. One is when you're setting it up, if you're using a geo test, if you're doing a geo test, then the impact, because of timing or seasonality, if you are able to roll out your geo test, which is in one market, so you changed your . . . I'm again going back to the example of the travel portal where TV was driving 50% of the visits. And it was getting 80% of the media spend. So the brand decided that they're going to
43:03
down-weight TV on the back of the model. But they decided that in two markets, which were comparable to begin with -- now, comparable to begin with: Again, how do you decide that? You look at the factors, like you look at a number of factors and you do some pre-analysis to say, okay, these two markets or these five markets are comparable to begin with. In three of these markets. I am going to go with plan A where TV is down-weighted. In other two markets, I'm going to go as is.
43:38
I'm going to do exactly what I would have done if I had not seen this model. And then you look at the incremental impact across the target markets and the control markets. Now, if you are doing it in this way, then things like AdStock, etc., all the lingering effects that we were spoken about would have already been taken care of.
44:02
So that's the right way to do it. But sometimes what happens is that one can only look at it post facto, like one doesn't get to set up the experiments in the right way and one gets results of the experiments, then there are technical ways to handle them. You can do like post facto correction on a limited number of things based on hypotheses that you have. So it's possible to do post correction, but it's best to set up these experiments so that you have a matched set of markets or you have . . .
44:39
Yeah, it's best to set these up so that you are going to get a more definite answer on the things that you're testing. And an experiment is, again, quite expensive because it only answers 1 or 2 questions. So that can't be
44:58
the only strategies that you have. MMM is what provides you with holistic answers, but like we looked at earlier, it is time consuming. It is resource intensive. And it also can give you . . . Like it's ultimately a correlation. Through incrementality, to establish that, and to establish causation, one has to go down the experiment path. If you want to get really granular, short-term, within-platform decisions, then despite the deprecation of third party cookies or the . . .
45:34
Google has recently gone back on it and it has said that they are going to go the Apple way where people can make a choice whether they want to be tracked or not tracked. Now, despite this decision, what's happening is that like
45:52
definitely, like what is universally accepted is that the third party cookies aren't going to be as powerful as they were earlier, and that the impact, the attribution, will have to be supplemented by other methods. So this trifecta of MMM, MTA and experiments being used to drive the right set of decisions for the right market and brands is what all brands need to do. And, depending on how much, like, what's the media spend, what's the channel mix and what's the business context within which
46:32
they want to make the decisions, they can pick the right methods and right metrics. That's interesting. I was looking forward to getting your opinion about the current state in regard to the cookies disappearing. So you're saying that they're really . . . it looks like they're not going to disappear, but they're going to be less impactful, and so we're going to have to evolve our systems in such a way that accounts for that. So I'll bet you've seen some very evolved measurement systems
47:03
at some of the top brands in the world, just out of curiosity, what does state of the art look like today?
47:12
I think what I will say is that there are principles that measurement systems need to have. So modularity, being future proof in the sense that it should not be dependent on any tools or technologies which are going to become less reliable or going to disappear altogether. Then it should be modular, because then modularity will ensure that you can get results from . . . you can get data sources from multiple systems. You can get results from multiple systems. And you can fuse the data sources
47:50
together to build a robust data set to begin with. And you can get results from multiple systems or multiple technologies to try to make sense of them. So modularity, future proof-ness, actionability. One also needs . . . Like being able to do it across the entire funnel is another principle that I would say is very important. If you're measurement system is only going to tell you about sales, but is not going to say anything about the awareness or consideration
48:23
or any of the upper-funnel metrics, then you are only going to look at a short term . . .
48:31
Your window is short term. You're not going to be able to answer questions like: What is the right amount of money to be spent on driving brand objectives? So, full funnel, actionable, futureproof, modular are some of the principles that we advocate. Yeah. Top of funnel is critical. Well, I'm so glad that we've had Tapan, a chance to reconnect again. I really enjoyed working with you. I remember a project that we did together for a mattress brand and identified some comparable markets,
49:05
so that if one wanted to do a test, we could split the two markets and make inference -- all of that work. If people wanted to get in touch with you, what's the best way for them to do that. So, either on LinkedIn or on email My email is tapan.khopkar@omd.com And I am also there on LinkedIn. And on LinkedIn. For those who don't necessarily know how to spell Khopkar, why don't you help me with that? Sure it is. K H O P (P for Paris) K A R All right. Tapan, this has been amazing.
49:48
I'm really grateful for having a chance to reconnect with you. Bye for now. Thank you, Jim. You've been listening to the AI Master Group podcast. I'm your host, Jim Griffin. Until next time, stay curious and stay in front.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat