1
00:00:00,000 –> 00:00:03,630
How much is media contributing relative
to customer base is a really nice place

2
00:00:03,630 –> 00:00:04,410
to start.

3
00:00:04,410 –> 00:00:09,240
And the benefit of running
incrementality and media mix modeling is

4
00:00:09,240 –> 00:00:12,000
informing the model with
some of that causal data.

5
00:00:24,780 –> 00:00:29,040
Well, hello and welcome to another edition
of the E-Commerce Evolution podcast.

6
00:00:29,280 –> 00:00:32,700
I’m your host, Brett
Curry, CEO of OMG Commerce.

7
00:00:32,700 –> 00:00:37,170
And today we have got
a doozy of an episode.

8
00:00:37,170 –> 00:00:42,030
We’re talking about the three
horsemen of measuring your

9
00:00:42,030 –> 00:00:45,900
marketing effectiveness. We’re
talking MTAs Multitouch attribution.

10
00:00:45,900 –> 00:00:49,470
We’re talking M’S. Media mixed
modeling. We’re talking incrementality.

11
00:00:49,740 –> 00:00:50,670
It’s going to be nerdy,

12
00:00:50,670 –> 00:00:55,530
but I also promise you it’s going to
be practical and it will make you more

13
00:00:55,740 –> 00:00:58,560
money. And so we’ll hopefully
make it fun as well.

14
00:00:58,560 –> 00:01:01,740
And so my guest today is Tom Leonard.

15
00:01:01,740 –> 00:01:04,290
We are LinkedIn friends first.

16
00:01:04,290 –> 00:01:09,270
So I saw Tom on LinkedIn posting about
incrementality, talking about MMM,

17
00:01:09,600 –> 00:01:13,650
throwing shade on certain tools and stuff
like that on LinkedIn. And I’m like,

18
00:01:14,010 –> 00:01:18,480
this is my type of guy. So I reached
out, we had a call, and then we’re like,

19
00:01:18,480 –> 00:01:20,370
Hey, we got to record a podcast.

20
00:01:20,370 –> 00:01:24,660
Let’s create some insights
for people on the pod.

21
00:01:25,050 –> 00:01:29,340
And so Tom is a fractional
marketing leader.

22
00:01:29,730 –> 00:01:33,630
He’s operationalizing MMM
and incrementality testing,

23
00:01:33,630 –> 00:01:37,590
and I’m delighted that he’s my guest
today. So Tom, with that intro,

24
00:01:37,890 –> 00:01:39,270
how’s it going? And welcome to the show.

25
00:01:39,780 –> 00:01:42,690
Good. Thanks for having me, Brent.
Excited to be here. And yeah,

26
00:01:42,810 –> 00:01:46,730
some of my favorite things to talk
through, so excited to do it. Good stuff.

27
00:01:46,830 –> 00:01:49,110
It’s good stuff, man. So briefly,

28
00:01:49,110 –> 00:01:51,870
before we dive into the
meat of the content here,

29
00:01:52,200 –> 00:01:56,220
what’s your background and
how did you become a guy who’s

30
00:01:56,520 –> 00:01:58,770
operationalizing MMS and incrementality?

31
00:01:58,840 –> 00:02:00,170
Yeah. And what does that even mean?

32
00:02:01,830 –> 00:02:02,690
That’s a good point.

33
00:02:02,710 –> 00:02:05,490
For sure. Yeah, totally. Yeah.

34
00:02:05,490 –> 00:02:09,990
So spent most of my career thus far on
the agency side at performance agencies.

35
00:02:10,830 –> 00:02:15,240
And I’d say the crux of
how I got to where I’m now,

36
00:02:15,240 –> 00:02:18,720
or I’ve been reflecting back a little
bit more on the why I have such a passion

37
00:02:18,720 –> 00:02:22,320
for measurement. And I was at
a pretty hardcore DR agency,

38
00:02:23,340 –> 00:02:28,050
and it was right shortly after TRUBY
for Action came out when YouTube was

39
00:02:28,050 –> 00:02:30,150
starting to invest in, DR.

40
00:02:31,350 –> 00:02:35,850
Moved into a new role we had created
with a centralized group of basically

41
00:02:35,850 –> 00:02:39,750
people who had different areas of subject
matter expertise and a few analysts

42
00:02:39,750 –> 00:02:42,920
that ran tests across a
pretty large client base.

43
00:02:42,920 –> 00:02:44,940
And I was our YouTube SME,

44
00:02:45,930 –> 00:02:48,600
and worked with a couple
analysts to run a bunch of tests.

45
00:02:48,840 –> 00:02:51,540
And really it was to evangelize how to,

46
00:02:51,540 –> 00:02:56,520
and is YouTube a platform to drive growth?
And it was really interesting

47
00:02:56,790 –> 00:02:59,800
because I started spending a lot of time
on YouTube and then also connect to TV

48
00:02:59,800 –> 00:03:04,150
and broader programmatic video.
And it was this really interesting,

49
00:03:04,780 –> 00:03:07,600
for me, the biggest learning was less
about how to make YouTube as effective as

50
00:03:07,600 –> 00:03:08,080
possible,

51
00:03:08,080 –> 00:03:12,400
but more how to help brands think about
demand creation as opposed to just

52
00:03:12,400 –> 00:03:13,900
demand capture. And frankly,

53
00:03:14,230 –> 00:03:19,090
the difficulty of getting brands
to leverage YouTube relative

54
00:03:19,090 –> 00:03:20,170
to connected tv,

55
00:03:20,170 –> 00:03:23,950
because YouTube sat so close to Google
ads and therefore last click attribution

56
00:03:24,220 –> 00:03:27,100
and see tv, you couldn’t click
and was sexier in a deck.

57
00:03:28,060 –> 00:03:31,450
And it was just this sort
of recognition of the

58
00:03:32,380 –> 00:03:37,330
irrational kind of human behavior just
in any sort of industry or any thing

59
00:03:37,330 –> 00:03:38,163
in life.

60
00:03:39,130 –> 00:03:43,930
But it sort of helped frame up this
idea of you really have to do more than

61
00:03:43,930 –> 00:03:45,880
just, I don’t know,

62
00:03:45,880 –> 00:03:50,680
represent logic or rational arguments.
You really have to also

63
00:03:50,680 –> 00:03:54,790
bring the easy to understand
clear data. And that’s,

64
00:03:54,790 –> 00:03:59,770
I think what draws me to incrementality
testing specifically and why

65
00:03:59,770 –> 00:04:02,380
that’s sort of the backbone
of a lot of what I do now.

66
00:04:02,380 –> 00:04:06,850
And I think I use the word
operationalizing, NMM and
incrementality testing.

67
00:04:06,850 –> 00:04:11,530
And really what I mean by that is a lot
of people will run medium mix models or

68
00:04:11,530 –> 00:04:12,880
run incrementality tests,

69
00:04:13,150 –> 00:04:17,290
but oftentimes they’ll sit in a slide
or in a report to be shown once,

70
00:04:17,290 –> 00:04:18,670
but never to be looked at again.

71
00:04:18,970 –> 00:04:22,870
And so what I’m really trying to do
with brands now is how do you build a

72
00:04:22,870 –> 00:04:26,620
framework and a repeatable methodology
to get insights from tests,

73
00:04:26,620 –> 00:04:29,470
but not just leave them as
insights but to take action?

74
00:04:29,500 –> 00:04:33,160
Because the only way that you create
value from any of these sort of testing

75
00:04:33,160 –> 00:04:36,820
methodologies and measurement
methodologies is by
acting on the insights.

76
00:04:37,630 –> 00:04:41,230
And so that’s sort of what I mean by my
funky little headline of those words.

77
00:04:41,710 –> 00:04:42,880
Yeah, it’s so good, man.

78
00:04:42,880 –> 00:04:46,360
And it’s one of those things where data
really doesn’t matter if you don’t take

79
00:04:46,360 –> 00:04:49,510
the right actions from it.
And what’s so interesting,

80
00:04:49,510 –> 00:04:54,430
and our paths are similar in that
I got my start in actually TV and

81
00:04:54,430 –> 00:04:57,940
radio and doing traditional media, and
then I got into SEO and paid search,

82
00:04:58,570 –> 00:05:02,710
but I loved video. Video was my
thing, but I love paid search as well.

83
00:05:02,710 –> 00:05:06,040
And then when TrueView and TrueView
for Action came out, I was like, whoa,

84
00:05:06,460 –> 00:05:07,840
these are all my world’s colliding.

85
00:05:07,840 –> 00:05:08,673
This is.

86
00:05:09,070 –> 00:05:11,350
Video and there’s some search components,

87
00:05:11,500 –> 00:05:14,830
at least some search intent involved
there. And it’s direct response.

88
00:05:14,830 –> 00:05:16,570
I’ve always been a direct response guy.

89
00:05:17,290 –> 00:05:19,990
I believe that marketing
should drive an outcome, right?

90
00:05:19,990 –> 00:05:22,840
Advertising should drive
a measurable outcome,

91
00:05:23,290 –> 00:05:26,350
and that should be measured in terms
of new customers and profitable new

92
00:05:26,350 –> 00:05:28,990
customer acquisition. And
what’s really interesting, Tom,

93
00:05:28,990 –> 00:05:31,150
and I think this kind of feeds into
the conversation we’re having today.

94
00:05:31,750 –> 00:05:35,350
There was a period of time, so I
grew up reading some of the classics.

95
00:05:35,350 –> 00:05:39,820
So David Ogilvy of course, but John
Cap’s tested advertising methods,

96
00:05:40,180 –> 00:05:42,160
Claude Hopkins Scientific Advertising.

97
00:05:42,160 –> 00:05:45,760
And they would do things like they would
run and add in a newspaper or magazine

98
00:05:45,760 –> 00:05:48,370
and people would clip a
coupon and bring it in,

99
00:05:48,370 –> 00:05:51,970
or they would call a certain number and
they would track it and they would have

100
00:05:51,970 –> 00:05:52,803
codes and stuff.

101
00:05:53,200 –> 00:05:58,040
And I remember thinking once I got
into e-commerce, I was like, oh man,

102
00:05:58,370 –> 00:06:03,170
we’ve got so many tools. The world is
so clear now we have every piece of

103
00:06:03,170 –> 00:06:04,790
data at our disposal.

104
00:06:05,030 –> 00:06:08,030
And now the more I’ve gotten into it
and the more I’ve matured, I’m like,

105
00:06:08,750 –> 00:06:11,120
we’ve got more data. But I don’t
know that we’ve got more insights,

106
00:06:11,120 –> 00:06:13,130
and I don’t know that we’ve
got any more clarity. In fact,

107
00:06:13,130 –> 00:06:14,420
there’s maybe more confusion.

108
00:06:14,420 –> 00:06:17,540
And I think it goes back to
what you said a minute ago,

109
00:06:17,840 –> 00:06:21,740
this idea of demand generation
versus demand capture.

110
00:06:22,100 –> 00:06:26,450
We’re really good at measuring channels
and campaigns that are demand capture,

111
00:06:26,450 –> 00:06:28,550
meaning they’re capturing
demand that’s already out there.

112
00:06:29,390 –> 00:06:31,700
That’s harder to measure
the demand generation,

113
00:06:31,880 –> 00:06:34,880
which is usually where the magic happens.

114
00:06:35,270 –> 00:06:37,670
And so super excited to dive in here.

115
00:06:37,670 –> 00:06:42,590
I think what might be useful
is let’s talk about what

116
00:06:42,710 –> 00:06:46,460
are these kind of three horsemen that
I laid out there, MTAs, multitouch,

117
00:06:46,460 –> 00:06:51,080
attribution, and incrementality.
So let’s start with MTAs first.

118
00:06:51,080 –> 00:06:54,020
So Multitouch attribution tools,

119
00:06:54,230 –> 00:06:56,420
what are they and what
is your take on them?

120
00:06:57,770 –> 00:07:01,580
Yeah, big question. Great
question. Yeah, I mean,

121
00:07:01,580 –> 00:07:04,520
MTA been around for a while,

122
00:07:04,820 –> 00:07:07,610
different flavors and ways
of trying to make it work,

123
00:07:07,790 –> 00:07:12,080
especially as so much has changed
in privacy and the tech and tracking

124
00:07:12,080 –> 00:07:12,800
landscape.

125
00:07:12,800 –> 00:07:16,280
But ultimately the goal is to try
to give fractional credit to all the

126
00:07:16,280 –> 00:07:20,810
touchpoints along a customer journey with
a recognition that the last touchpoint

127
00:07:22,010 –> 00:07:26,930
click or last impression is
ultimately not what drove that person

128
00:07:26,930 –> 00:07:27,950
to purchase.

129
00:07:27,950 –> 00:07:32,930
That may be the last or the only thing
that you might see in something like

130
00:07:32,930 –> 00:07:35,960
Google Analytics or your analytics suite.

131
00:07:36,830 –> 00:07:40,640
But there’s this general recognition
that that is not what drove the purchase.

132
00:07:40,640 –> 00:07:45,110
So MTA, the kind of promise, which I
ultimately think is a failed promise,

133
00:07:45,110 –> 00:07:50,090
is whether all the different touch
touchpoint and then how can you

134
00:07:50,480 –> 00:07:53,270
value those differently. So
maybe you use first touch,

135
00:07:53,270 –> 00:07:57,830
maybe you use even distribution. The
idea of data-driven attribution was the

136
00:07:57,830 –> 00:07:59,870
holy rail or the Promise many years ago,

137
00:08:00,320 –> 00:08:03,370
and I guess still to a
degree for some is like,

138
00:08:03,370 –> 00:08:07,700
how do you know this channel was more
additive or more necessary and therefore

139
00:08:07,700 –> 00:08:10,580
should get more credit than that channel?

140
00:08:11,690 –> 00:08:13,670
Which I think makes a
ton of sense in promise.

141
00:08:14,630 –> 00:08:19,520
I think in reality it’s really hard
and I would argue impossible to do,

142
00:08:19,520 –> 00:08:23,870
especially as a lot of the ability to
track users at a one-to-one level degrades

143
00:08:24,680 –> 00:08:27,770
generally my perspective,
I’m very bearish on MTA,

144
00:08:27,950 –> 00:08:29,810
so that’ll probably come
through pretty strongly.

145
00:08:30,410 –> 00:08:33,110
But I guess I don’t think the toothpaste
is going back in the tube in terms of

146
00:08:33,110 –> 00:08:36,620
the ability to track a customer across
all these different touchpoints,

147
00:08:37,790 –> 00:08:42,500
especially as the ability to
track through or impression based

148
00:08:42,500 –> 00:08:47,240
touchpoint erodes. And then you
really get reliant on clicks,

149
00:08:47,240 –> 00:08:50,780
which I think then leads to a lot of
all the issues that just last click in

150
00:08:50,780 –> 00:08:51,613
general has.

151
00:08:52,580 –> 00:08:57,090
So I think it’s really hard to
make a compelling case for MTA.

152
00:08:57,090 –> 00:08:59,490
I’ve seen too many brands,

153
00:08:59,490 –> 00:09:02,190
especially trying to
build MTA tools internally

154
00:09:03,750 –> 00:09:08,130
and just be a huge time and resource
suck. And then when you ask to compare,

155
00:09:08,670 –> 00:09:12,570
show the multi-touch view versus
last click, it’s like, I don’t know,

156
00:09:12,600 –> 00:09:17,130
80 or 90% only had one touch
point anyways, that’s all
that MTA model could see.

157
00:09:17,310 –> 00:09:20,580
So is it really that much
more useful than last click?

158
00:09:22,320 –> 00:09:26,580
It’s sort of multi-touch when that can
be measured, but usually it can’t be.

159
00:09:26,580 –> 00:09:30,720
Yeah, and It never really answers
the causality question either,

160
00:09:30,720 –> 00:09:32,650
which we’ll get to when we
talk about incrementality.

161
00:09:32,670 –> 00:09:33,840
And I always kind of tell this,

162
00:09:34,860 –> 00:09:39,510
I think the short story of why MT A
isn’t really viable anymore as all the

163
00:09:39,510 –> 00:09:41,040
tracking and privacy changes.

164
00:09:41,580 –> 00:09:44,910
But I think the slightly longer story
is the kind of recognition that just

165
00:09:44,910 –> 00:09:49,320
because an ad was shown or a
click occurred doesn’t mean that

166
00:09:49,680 –> 00:09:52,380
that medium was needed or
that channel was needed.

167
00:09:52,380 –> 00:09:54,210
It doesn’t answer the causal question,

168
00:09:54,210 –> 00:09:56,100
what would’ve happened
without this ad running?

169
00:09:56,100 –> 00:10:00,210
Did somebody just happen to use multiple
touchpoints as navigation or was it

170
00:10:00,210 –> 00:10:03,390
more convenient to click on one of
these ads that happened to be served?

171
00:10:03,390 –> 00:10:08,250
But if you’re not comparing that to some
sort of control group to really hard

172
00:10:08,700 –> 00:10:11,490
to assign causality to the fact
that there just was a touchpoint.

173
00:10:12,750 –> 00:10:16,530
Yeah, it is so good. And it’s one of
those things where I remember again,

174
00:10:16,530 –> 00:10:17,280
early on,

175
00:10:17,280 –> 00:10:20,890
you would look inside of Google ads or
you look inside of Meta or was back when

176
00:10:20,890 –> 00:10:24,630
it was Facebook only, and you
were like, the data’s here.

177
00:10:24,630 –> 00:10:27,300
I see row ads and I see clicks and
I see performance and all that.

178
00:10:28,380 –> 00:10:30,630
Then you realize, well, wait a
minute, this isn’t fully accurate.

179
00:10:30,630 –> 00:10:33,900
If I add the two together,
that’s double my total revenue,

180
00:10:34,110 –> 00:10:36,030
so I can’t just rely on
what’s in the platform.

181
00:10:36,030 –> 00:10:40,170
And that got worse as I was 14 was
introduced and other privacy changes were

182
00:10:40,170 –> 00:10:44,010
made. But then MTA came
along and it’s like, oh,

183
00:10:44,010 –> 00:10:48,120
finally we’re going to get to see the
full picture. It’s going to decipher,

184
00:10:48,120 –> 00:10:50,940
decode the shopping journey,

185
00:10:50,940 –> 00:10:55,770
and we’re going to finally see with a
keen eye in perfection exactly what caused

186
00:10:55,770 –> 00:11:00,540
this ad or what caused this purchase
to happen. And then we finally realized

187
00:11:01,650 –> 00:11:04,710
MTA is maybe just a third
option. It’s like, okay,

188
00:11:04,710 –> 00:11:09,420
Google’s imperfect, Meta’s
data’s imperfect, and then mt A,

189
00:11:09,420 –> 00:11:10,410
it’s just imperfect too.

190
00:11:10,410 –> 00:11:15,330
So now we just got three imperfect
things to look at and make

191
00:11:15,360 –> 00:11:16,230
decisions from.

192
00:11:16,230 –> 00:11:20,190
And in some ways it leads to more
confusion than it leads to clarity.

193
00:11:20,640 –> 00:11:25,290
And now I don’t want to wholesale discard

194
00:11:25,290 –> 00:11:28,950
MTAs because I do believe there’s some
helpful insights that can be gained

195
00:11:28,950 –> 00:11:29,783
there,

196
00:11:30,840 –> 00:11:34,200
but it’s incomplete
and incomplete at best.

197
00:11:34,200 –> 00:11:37,260
And one of the best analogies I’ve heard,
and this actually comes from Ben Ter,

198
00:11:38,490 –> 00:11:41,400
who’s also a LinkedIn friend,
but I met him in person as well,

199
00:11:41,820 –> 00:11:43,470
but he talks about this analogy of, Hey,

200
00:11:43,470 –> 00:11:48,030
if we’re trying to measure what
caused people to watch this

201
00:11:48,030 –> 00:11:49,860
movie at our movie theater,

202
00:11:50,220 –> 00:11:55,030
and we look at all these
results and 30% say they saw a

203
00:11:55,030 –> 00:11:59,170
billboard for our movies,
20% say they saw a TV ad,

204
00:11:59,380 –> 00:12:03,880
but you know what? A hundred percent
say they saw the poster on the

205
00:12:03,880 –> 00:12:06,940
door. So we’re like,
let’s just cut everything.

206
00:12:06,940 –> 00:12:09,760
Let’s just do the poster at the door
and that’s it. And you’re like, well,

207
00:12:09,760 –> 00:12:13,000
wait a minute. Everybody saw it.
Everybody was walking in the door.

208
00:12:13,300 –> 00:12:16,990
But the movie poster is not
what caused someone to purchase.

209
00:12:17,440 –> 00:12:19,720
It was the billboard and the TV
and some of the other things,

210
00:12:19,720 –> 00:12:21,760
word of mouth and other things
that caused them to come in.

211
00:12:21,760 –> 00:12:26,620
And so this idea of causality,
super, super valuable.

212
00:12:26,620 –> 00:12:31,480
So that really leads us to incrementality.
So talk about incrementality.

213
00:12:31,780 –> 00:12:35,680
What is it and why are you on
a quest to operationalize it?

214
00:12:37,060 –> 00:12:40,210
Yeah, it’s really the best way,

215
00:12:40,210 –> 00:12:44,380
if not the only way to
establish that a causal

216
00:12:44,920 –> 00:12:49,570
portion that we’ve been talking about.
It has a distinct control group,

217
00:12:49,570 –> 00:12:51,190
so it has a counterfactual,

218
00:12:51,370 –> 00:12:54,340
it has what would’ve happened
without this intervention,

219
00:12:54,340 –> 00:12:56,200
whatever that intervention is.

220
00:12:56,200 –> 00:13:00,820
And there’s a handful of ways to derive
that counterfactual that control.

221
00:13:00,820 –> 00:13:04,570
The most common would be geographic
based. So like a match market test.

222
00:13:04,720 –> 00:13:08,530
I’ve got this market over here that
historically has behaved similarly to this

223
00:13:08,530 –> 00:13:11,500
market over here. I can
see that in an AA test,

224
00:13:11,500 –> 00:13:14,080
the lines sort of move similar
to one another. They’re not,

225
00:13:14,740 –> 00:13:17,050
if they’re influenced by outside
factors, they’re influenced.

226
00:13:17,050 –> 00:13:18,580
In what’s an AA test for
those who don’t know.

227
00:13:19,090 –> 00:13:20,590
Before an intervention happens.

228
00:13:20,620 –> 00:13:24,130
So just over time are those lines
essentially moving together?

229
00:13:24,610 –> 00:13:29,320
Are external factors or stimuli equally
impacting both sides of that test

230
00:13:29,650 –> 00:13:33,490
so that you can feel confident that
when you do intervene and it becomes

231
00:13:33,490 –> 00:13:34,720
comparing A to B,

232
00:13:35,320 –> 00:13:39,400
the delta is what was a
result of that intervention.

233
00:13:39,400 –> 00:13:44,050
So oftentimes it’s my Atlanta

234
00:13:44,110 –> 00:13:45,760
and I don’t know Memphis,

235
00:13:45,760 –> 00:13:49,780
maybe some other midsize city that
you’ve done this market matching for.

236
00:13:50,230 –> 00:13:52,390
Historically, they both
look like this on a line,

237
00:13:52,450 –> 00:13:55,840
all of a sudden you turn off
ads on Facebook in Atlanta,

238
00:13:55,870 –> 00:14:00,700
what happens to your top line that
Delta is what was attributed or

239
00:14:00,700 –> 00:14:04,810
should be attributed to
advertising in Atlanta.

240
00:14:05,110 –> 00:14:08,590
Whereas the flip side of that would be
attribution would say basically anything

241
00:14:08,590 –> 00:14:12,130
that was attributed to that could
be attributed to that would really,

242
00:14:12,250 –> 00:14:16,630
it should just be the gap between a
world where that ad does not exist

243
00:14:16,900 –> 00:14:19,810
compared to a world where that ad
does exist. We can’t take credit for

244
00:14:19,810 –> 00:14:20,643
everything.

245
00:14:20,890 –> 00:14:24,460
We can only take credit for as much
above and beyond what would’ve happened

246
00:14:24,460 –> 00:14:28,840
anyways. And so that’s the
basis of incrementality testing.

247
00:14:28,840 –> 00:14:30,010
There’s other ways to do it.

248
00:14:30,910 –> 00:14:35,770
If you use a Facebook or Google
conversion lift study because they own

249
00:14:35,770 –> 00:14:38,380
that auction or anybody
that owns an auction,

250
00:14:38,380 –> 00:14:41,080
they can do that hold out
for you at a user level.

251
00:14:41,260 –> 00:14:45,130
They can track all of those users
regardless of if you serve an ad.

252
00:14:46,420 –> 00:14:51,410
Good examples are maybe easier to
describe in a first party data capacity.

253
00:14:51,410 –> 00:14:55,670
If you’re running email, you may blast
all of your customers and say, Hey,

254
00:14:55,970 –> 00:14:58,970
I sent an email to all my
customers and this many purchased.

255
00:14:59,210 –> 00:15:03,110
They went back to the website or
clicked it. But if you just said, Hey,

256
00:15:03,110 –> 00:15:07,850
I’m going to serve just to odd
number of customer IDs and not to

257
00:15:07,850 –> 00:15:11,240
even number customer IDs,
I can then just compare,

258
00:15:11,240 –> 00:15:12,650
forget about who clicked on ads,

259
00:15:12,650 –> 00:15:14,390
who did anything.
I’m just going to look at my backend.

260
00:15:14,660 –> 00:15:18,530
I know I exposed these users,
but not these users 50 50 split.

261
00:15:18,620 –> 00:15:20,360
They’ve historically kind
of done the same thing.

262
00:15:20,360 –> 00:15:24,080
All I did was even an odd and just
measuring the difference between those two

263
00:15:24,080 –> 00:15:24,440
groups.

264
00:15:24,440 –> 00:15:29,150
So really any way that you can
establish a true control that

265
00:15:29,150 –> 00:15:33,590
passes that AA test. So
before you intervene, do they
continue to look similar?

266
00:15:33,770 –> 00:15:37,190
Are they influenced at the same rate so
that you can feel confident that when

267
00:15:37,190 –> 00:15:41,270
you do intervene with new
media, retracting media,

268
00:15:41,510 –> 00:15:46,040
some new sort of test that you are
confidently comparing to what would’ve

269
00:15:46,040 –> 00:15:47,960
happened in a world
without that intervention?

270
00:15:48,350 –> 00:15:49,010
Yeah, yeah.

271
00:15:49,010 –> 00:15:53,570
It’s applying the scientific
method with some rigor behind

272
00:15:53,870 –> 00:15:56,750
what happens when I turn this channel on,

273
00:15:57,110 –> 00:15:59,480
or what happens when I
turn this channel off?

274
00:15:59,840 –> 00:16:02,540
What is the actual impact of this channel?

275
00:16:02,540 –> 00:16:07,100
And what’s interesting is I
remember back in my early days

276
00:16:07,130 –> 00:16:09,770
of being in the advertising world,

277
00:16:09,770 –> 00:16:12,290
this was when online stuff was
just getting kind of warmed up.

278
00:16:12,590 –> 00:16:16,010
I was talking to this furniture store
owner and I’m like, Hey, what do you do?

279
00:16:16,100 –> 00:16:19,700
Do you invest in radio ads?
Do tv, do you do newspaper?

280
00:16:19,910 –> 00:16:21,800
And so as I went through Themm like,
Hey, do you do radio ads? And he is like,

281
00:16:21,800 –> 00:16:25,580
yeah, I mean, yeah, I sort of do.
And I’m like, newspaper’s like, yeah,

282
00:16:26,180 –> 00:16:28,670
there’s a big sale, something will
happen. I’m like, well, what about tv?

283
00:16:28,670 –> 00:16:31,550
And he said, yes. And his
eyes lit up and he is like,

284
00:16:31,550 –> 00:16:36,260
when I run TV ads, I feel
it. People walk in the door,

285
00:16:36,260 –> 00:16:40,430
it happens. And I remember early on
in my online career thinking, man,

286
00:16:40,430 –> 00:16:43,070
that was so unsophisticated. Did
that guy really know what’s going on?

287
00:16:43,340 –> 00:16:46,100
But now looking back, I’m like,
yeah, that’s maybe all that matters.

288
00:16:46,820 –> 00:16:51,810
That is incrementality in a real loose
easy just to observe with your eyes think

289
00:16:51,810 –> 00:16:52,650
because you had one. Totally.

290
00:16:53,180 –> 00:16:55,410
Which I think people
take for granted. Yeah.

291
00:16:55,410 –> 00:16:56,243
They do.

292
00:16:56,540 –> 00:16:56,990
Yeah.

293
00:16:56,990 –> 00:16:59,240
That’s not exciting. That’s not
like, where’s all your data?

294
00:16:59,960 –> 00:17:02,150
It’s in my cash register.
That’s where all the data.

295
00:17:02,150 –> 00:17:04,670
Is, especially for smaller brands,

296
00:17:05,540 –> 00:17:10,310
when you have the ability
to feel if something’s

297
00:17:10,310 –> 00:17:11,360
working or not working,

298
00:17:11,540 –> 00:17:15,230
if you double spend in something that
you think is working really well because

299
00:17:15,230 –> 00:17:16,910
attribution says it’s working really well,

300
00:17:17,180 –> 00:17:18,770
and all of a sudden
your cash just doubles,

301
00:17:18,770 –> 00:17:22,730
even though your attributed number
scales linearly, something has to give,

302
00:17:22,970 –> 00:17:23,300
right?

303
00:17:23,300 –> 00:17:27,950
And what has to give is it wasn’t really
causing any additional top line growth.

304
00:17:27,950 –> 00:17:30,230
It was just really good at
getting the attributed credit.

305
00:17:30,530 –> 00:17:34,010
So I think the feeling
it in the p and l is

306
00:17:35,180 –> 00:17:36,020
definitely overlooked.

307
00:17:36,800 –> 00:17:39,020
It’s valid, and it is overlooked
though. You’re a hundred percent,

308
00:17:39,170 –> 00:17:40,940
especially now that we have
so many tools at our disposal.

309
00:17:40,940 –> 00:17:43,880
And I think another way to look at
this, and look, I’m a Google guy,

310
00:17:44,090 –> 00:17:48,890
YouTube and Google is kind of where
I really got my start in online.

311
00:17:48,900 –> 00:17:49,733
Marketing.

312
00:17:49,950 –> 00:17:53,160
But listen, branded search is a
perfect example here. What happens,

313
00:17:53,160 –> 00:17:53,970
we see this all the time.

314
00:17:53,970 –> 00:17:58,050
What happens if you turn branded
search completely off? Now, I believe,

315
00:17:58,050 –> 00:18:00,000
and this is top of front of the podcast,

316
00:18:00,240 –> 00:18:03,510
there are strategic ways to use branded
search and there’s ways to run it and

317
00:18:03,510 –> 00:18:07,890
not waste money, but a lot of people
could shut it off and nothing happens,

318
00:18:07,950 –> 00:18:10,020
nothing. Maybe sales get in a little bit,

319
00:18:10,020 –> 00:18:14,130
but you take meta meta’s really working
and you shut it off and you feel it.

320
00:18:14,400 –> 00:18:17,970
Sales go down and that’s
an incrementality.

321
00:18:17,970 –> 00:18:22,140
Same is true for YouTube if you’re doing
YouTube the right way. And so yeah,

322
00:18:22,140 –> 00:18:26,250
I really like this. And one
kind of anecdote here to share,

323
00:18:26,250 –> 00:18:31,050
we just did a test with Arctic,
Arctic coolers, Yeti competitor,

324
00:18:31,290 –> 00:18:35,190
my favorite cooler, my favorite drinkware
as well. And so they wanted to see,

325
00:18:35,190 –> 00:18:40,170
Hey, can YouTube drive an incremental
lift at Walmart? So they had just

326
00:18:40,230 –> 00:18:42,690
gotten into most Walmart
stores, coast to coast.

327
00:18:43,050 –> 00:18:47,940
So we did exactly what you laid out
there. We had a 19 test markets,

328
00:18:48,180 –> 00:18:51,600
19 matched control markets.
So similar markets.

329
00:18:51,600 –> 00:18:54,480
So think like a Denver and a
Kansas City or the example,

330
00:18:54,480 –> 00:18:57,750
use Atlanta and whatever else
that’s kind of comparable. And hey,

331
00:18:57,750 –> 00:19:00,180
let’s run YouTube in one
and not in the other.

332
00:19:00,210 –> 00:19:03,480
And let’s measure then the
growth in Walmart sales,

333
00:19:03,480 –> 00:19:05,970
and let’s do a comparison
between the two in Walmart sales.

334
00:19:05,970 –> 00:19:08,760
And it was remarkable. It
was about an eight week test.

335
00:19:09,210 –> 00:19:13,470
We had three test regions, so 19
markets, but three test regions,

336
00:19:14,040 –> 00:19:19,020
test region. One, we saw an average
of 12% lift in Walmart sales.

337
00:19:19,800 –> 00:19:23,940
The test region two was like 15% lift.

338
00:19:23,940 –> 00:19:28,020
And then our final test
region was 25% lift.

339
00:19:28,020 –> 00:19:29,400
And there were some standouts,

340
00:19:29,400 –> 00:19:33,990
like Oklahoma City was up 40% and Salt
Lake City was up 48%. But it was one of

341
00:19:33,990 –> 00:19:37,290
those things where, okay, now we
look at that and we can say, okay,

342
00:19:37,350 –> 00:19:40,110
YouTube had a big impact. And
what’s also interesting, Tom,

343
00:19:40,110 –> 00:19:42,450
is we just ran the YouTube portion at OMG.

344
00:19:42,900 –> 00:19:47,130
They also did a connected TV test
in other markets, not related,

345
00:19:47,820 –> 00:19:49,920
didn’t see a lift, didn’t
see a measurable lift.

346
00:19:49,920 –> 00:19:54,570
And so it could be lots of
that was not to throw shade on

347
00:19:54,570 –> 00:19:56,490
CTVI like CTV,

348
00:19:56,550 –> 00:19:59,100
so maybe they just did a wrong or
wrong creatives or who knows what.

349
00:19:59,100 –> 00:20:01,350
But it’s one of those things
where it’s like, okay,

350
00:20:01,350 –> 00:20:04,050
if you do this the right way,
you should see an impact.

351
00:20:05,350 –> 00:20:07,410
And I think touching on the
piece that I didn’t mention,

352
00:20:07,410 –> 00:20:11,460
the other beauty or value of
incrementality testing relative to

353
00:20:11,820 –> 00:20:16,650
attribution or mt a is the ability
to see beyond your.com to be able to

354
00:20:16,650 –> 00:20:20,970
see what’s happening on third parties
like Amazon, what’s happening in store.

355
00:20:20,970 –> 00:20:24,600
If you get that data own an operated
store or if you can get that through

356
00:20:24,600 –> 00:20:27,870
wholesale data, it really simplifies.

357
00:20:28,620 –> 00:20:30,870
There’s so much complexity.
And I think that’s, again,

358
00:20:30,870 –> 00:20:33,930
one of the rubs that I have
with MTA is all of them,

359
00:20:35,130 –> 00:20:38,790
all of the data you have to
wrangle together to try to

360
00:20:39,810 –> 00:20:42,060
patchwork this kind of story together.

361
00:20:42,990 –> 00:20:45,870
Whereas in incrementality testing,
it’s pretty straightforward.

362
00:20:45,870 –> 00:20:50,860
It’s what did I spend and how
did I run that spend in these by

363
00:20:50,860 –> 00:20:55,390
market by day or by week, and what
was my sales? What were my sales?

364
00:20:55,630 –> 00:21:00,370
What were my new customers or whatever
metric I’d want to look at with that same

365
00:21:00,370 –> 00:21:02,650
granularity and same dimension.

366
00:21:04,030 –> 00:21:06,550
And that’s really it because you’re
really just trying to understand the

367
00:21:06,550 –> 00:21:10,690
relationship that calls the
relationship between spend and outcomes,

368
00:21:11,740 –> 00:21:15,100
all that kind of muddy middle
in the middle, trying to
get it at the user level,

369
00:21:15,100 –> 00:21:19,240
which again, not going back into
the tube really simplifies things.

370
00:21:19,720 –> 00:21:20,710
Yeah, it does.

371
00:21:20,710 –> 00:21:23,770
And another thing that was
kind of interesting that
came a light doing this test

372
00:21:23,770 –> 00:21:28,660
for Arctic is all of the ads we
tagged with available at Walmart,

373
00:21:28,810 –> 00:21:32,260
shop at Walmart, find on the
shelves and Walmart, whatever.

374
00:21:34,360 –> 00:21:36,430
We measured everything
though in those markets.

375
00:21:36,610 –> 00:21:40,780
So you could look at Walmart sales,
online sales, so the.com and Amazon.

376
00:21:40,910 –> 00:21:43,930
And what’s interesting is the
push to Walmart really worked.

377
00:21:44,110 –> 00:21:48,820
It’s a reminder of what you ask someone
to do in an ad is what they’re going to

378
00:21:48,820 –> 00:21:51,070
lean towards. Because
in some of the markets,

379
00:21:51,070 –> 00:21:52,720
we didn’t see that much of an online lift.

380
00:21:52,720 –> 00:21:56,110
We saw some clicks and stuff like
that, but the Lyft was at Walmart.

381
00:21:56,980 –> 00:21:59,380
But we also saw a pretty
strong lift at Amazon as well,

382
00:21:59,380 –> 00:22:00,820
because I think that just speaks to,

383
00:22:01,210 –> 00:22:03,400
there’s some people that are just going
to buy everything from Amazon right

384
00:22:03,400 –> 00:22:07,810
there, tell ’em to go online value pro
proposition. Is it on Amazon? Yeah, yeah.

385
00:22:08,560 –> 00:22:10,990
Yeah. Here in a day or two, it’s hard.

386
00:22:10,990 –> 00:22:14,560
To beat, dude. It’s hard to beat
same price in a couple days.

387
00:22:14,560 –> 00:22:19,060
I don’t have to leave my house. But
yeah, really, really interesting.

388
00:22:19,720 –> 00:22:22,060
And so we’ll circle
back to that of course,

389
00:22:22,060 –> 00:22:26,080
but let’s talk about then
MMM or media mix modeling.

390
00:22:26,500 –> 00:22:29,320
What is that? How are you using that?

391
00:22:29,320 –> 00:22:33,370
And then how does that kind of relate to
incrementality testing? Because again,

392
00:22:33,370 –> 00:22:37,770
going back to your tagline, Tom, you
did not say operationalizing NTAs.

393
00:22:38,110 –> 00:22:40,840
You said operationalizing m
and ms and incrementality.

394
00:22:40,840 –> 00:22:44,440
So what is MM and how does
that pair with incrementality?

395
00:22:44,590 –> 00:22:44,950
Yeah,

396
00:22:44,950 –> 00:22:49,510
basically a big correlation exercise
trying to suss out without a true kind of

397
00:22:49,510 –> 00:22:50,380
holdout group,

398
00:22:50,380 –> 00:22:55,270
what is the impact and contribution of
each media channel and also what would

399
00:22:55,270 –> 00:22:56,350
happen without media.

400
00:22:56,350 –> 00:22:59,830
So trying to suss out a lot of the
same questions as incrementality,

401
00:22:59,830 –> 00:23:03,970
but basically using correlation as
opposed to having a true holdout group.

402
00:23:04,930 –> 00:23:06,160
So basically,

403
00:23:06,160 –> 00:23:10,870
and I’m sure all the hardcore MMM people
and data scientists will thumbs down

404
00:23:12,040 –> 00:23:16,540
this or whatever you can do to podcast,
but hey, in this period of time,

405
00:23:17,290 –> 00:23:20,380
sales went up and nothing could really
explain that other than the fact that

406
00:23:20,380 –> 00:23:25,180
TikTok spend went up and essentially
doing that at a mass scale over longer

407
00:23:25,180 –> 00:23:29,500
periods of time trying to take into
account anything that could explain that.

408
00:23:29,500 –> 00:23:33,130
So you’ll always kind of flag it with
these are promotions that happen,

409
00:23:33,130 –> 00:23:35,980
it should because you’re going to give
a model at least like two years worth of

410
00:23:35,980 –> 00:23:37,390
data or two years worth of data,

411
00:23:38,170 –> 00:23:41,080
it’ll bring in seasonality and try to
understand those sort of trends. So it’s

412
00:23:41,080 –> 00:23:44,740
trying to pull out if not
seasonality, if not promotions,

413
00:23:44,740 –> 00:23:47,480
if not some other things
that we are flagging.

414
00:23:48,560 –> 00:23:52,340
And it wasn’t price reductions,
it wasn’t all these pieces,

415
00:23:52,490 –> 00:23:55,130
what was happening in media
that could explain that change.

416
00:23:55,310 –> 00:23:58,610
And so that’s ultimately
what MMM is doing.

417
00:23:58,820 –> 00:24:00,410
It’s a big correlation exercise,

418
00:24:00,410 –> 00:24:05,240
figuring out roughly what is the channel
contribution to a top line revenue or

419
00:24:05,240 –> 00:24:07,550
order number and what’s really important.

420
00:24:07,550 –> 00:24:12,440
I think the nicest part or the best
first step with M is trying to get an

421
00:24:12,440 –> 00:24:13,970
understanding of a base,

422
00:24:14,000 –> 00:24:17,390
which is what it’s going to be called or
intercept what without the presence of

423
00:24:17,390 –> 00:24:17,810
ads,

424
00:24:17,810 –> 00:24:22,550
does this model think that my sales would
be such that I can then calculate not

425
00:24:23,150 –> 00:24:27,410
a total CAC of just looking at
total new customers divided by cost,

426
00:24:27,410 –> 00:24:32,240
but incremental to media
or remove base from

427
00:24:32,240 –> 00:24:33,080
that equation,

428
00:24:33,290 –> 00:24:37,550
how many conversions were contributed
because of media as this model sees,

429
00:24:37,550 –> 00:24:39,500
which no model is going to be perfect,

430
00:24:39,500 –> 00:24:41,480
no measurement method
is going to be perfect,

431
00:24:41,480 –> 00:24:43,280
but it’s a really nice
place to start to say,

432
00:24:43,940 –> 00:24:47,690
I knew I couldn’t account all
new customers to advertising,

433
00:24:47,690 –> 00:24:51,170
but what’s a good number to use or
to start with? Well, it looks like,

434
00:24:51,740 –> 00:24:55,310
and this will depend on the maturity of
the brand, but a really mature brand,

435
00:24:55,310 –> 00:24:56,360
I mean super mature brand,

436
00:24:56,360 –> 00:25:01,310
the big CPGs might be like 99% base
smaller brand might be something

437
00:25:01,310 –> 00:25:03,980
like 50% because you’ve got
this word of mouth flywheel,

438
00:25:03,980 –> 00:25:05,240
you’ve got product market fit,

439
00:25:05,390 –> 00:25:09,380
but trying to get an understanding of how
much is media contributing relative to

440
00:25:09,380 –> 00:25:11,390
customer base is a really
nice place to start.

441
00:25:11,660 –> 00:25:16,520
And the benefit of running
incrementality and media mix modeling is

442
00:25:16,520 –> 00:25:19,280
informing the model with
some of that causal data.

443
00:25:19,730 –> 00:25:24,320
You see that a lot and there’s a
really powerful feature of media mix

444
00:25:24,320 –> 00:25:27,650
modeling is saying, Hey, yes,
that’s a correlation exercise,

445
00:25:28,580 –> 00:25:29,780
can’t pull everything out,

446
00:25:30,260 –> 00:25:33,920
but let me inform the model or at least
restrict the priors it can use or the

447
00:25:33,920 –> 00:25:35,660
coefficient, whatever
you want to call ’em,

448
00:25:36,110 –> 00:25:39,590
what it’s searching for to try to find
a fit in this model and say, well,

449
00:25:39,590 –> 00:25:42,170
I did a hold out test. I know
you don’t have the causal data,

450
00:25:42,170 –> 00:25:46,340
but we ran this in this channel and that
channel and helping that restrict the

451
00:25:46,340 –> 00:25:50,750
model and giving it data that it can’t
have without that human intervention can

452
00:25:50,750 –> 00:25:52,130
be a really powerful flywheel.

453
00:25:52,790 –> 00:25:55,460
So using your incrementality test data,

454
00:25:55,460 –> 00:25:59,960
feeding that back into your MMM
model to make it more accurate and

455
00:26:00,440 –> 00:26:02,930
more causal and make that correlation.

456
00:26:02,930 –> 00:26:03,763
Stronger.

457
00:26:03,770 –> 00:26:07,250
Because the two things that are really
like you’re really trying to get,

458
00:26:07,430 –> 00:26:11,030
but you don’t get with Multi-Tech
attribution or attribution in general.

459
00:26:11,030 –> 00:26:14,840
And you do get with the combination of
media mix modeling and incrementality

460
00:26:14,840 –> 00:26:17,150
testing is the incremental impact,

461
00:26:17,150 –> 00:26:20,960
the causal impact of what
would’ve happened without
the presence of ads as well

462
00:26:20,960 –> 00:26:22,340
as the diminishing returns curve,

463
00:26:22,340 –> 00:26:24,740
which we know can be really
powerful and important too,

464
00:26:25,130 –> 00:26:29,450
is what has happened over time as I
spend in that sort of a feature of big

465
00:26:29,450 –> 00:26:33,050
feature of media mix modeling
is understanding where
are you on a diminishing

466
00:26:33,050 –> 00:26:35,570
returns curve? Is there
if I keep spending more,

467
00:26:35,570 –> 00:26:37,310
I know it’s not going to scale linearly,

468
00:26:37,550 –> 00:26:39,440
but are there channels
that diminish faster?

469
00:26:39,440 –> 00:26:41,600
Is there more headroom in other channels?

470
00:26:41,600 –> 00:26:46,020
And it really becomes this
true optimization game of
where do I put the next

471
00:26:46,020 –> 00:26:48,870
dollar? Ultimately the
question that every marketer,

472
00:26:48,870 –> 00:26:51,600
every finance team is
trying to answer is, Hey,

473
00:26:51,600 –> 00:26:54,900
if I find $20,000 into couch
cushions, where do I put it?

474
00:26:55,260 –> 00:27:00,150
And if I need to give back $20,000,
where do I pull from to have.

475
00:27:01,140 –> 00:27:04,110
I want to hang out at your house and
look at your couch cushions and find 20

476
00:27:04,110 –> 00:27:04,680
grand? That’s.

477
00:27:04,680 –> 00:27:08,910
Great. Yeah, it’s easy to
give it back, but yeah, right.

478
00:27:08,910 –> 00:27:11,820
We’re trying to figure out what is going
to be the least impactful if I have to

479
00:27:11,820 –> 00:27:15,990
give the money back and cut budgets
and where is it going to be the most

480
00:27:15,990 –> 00:27:18,270
impactful if I have another $20,000?

481
00:27:18,270 –> 00:27:21,810
Because the answer is not going to be
found in what has the highest or the

482
00:27:21,810 –> 00:27:25,440
lowest ROAS in an attributed
view. And in fact,

483
00:27:25,440 –> 00:27:28,950
that can have the complete
opposite impact that you want.

484
00:27:29,400 –> 00:27:30,990
Yeah, yeah, it’s really great.

485
00:27:30,990 –> 00:27:34,950
So I want to actually talk about
that point in a minute where

486
00:27:35,850 –> 00:27:38,490
if you’ve got cut budgets,
which hey, listen,

487
00:27:38,490 –> 00:27:42,120
there’s been some uncertainty even as we
record this, tariffs up, tariffs down,

488
00:27:43,110 –> 00:27:46,980
markets up, market down, whatever
consumer sentiment is all over the place.

489
00:27:47,340 –> 00:27:50,070
So if things get a little bit
tight, what are we going to do?

490
00:27:50,070 –> 00:27:53,100
We can’t slash marketing,
we can’t slash growth.

491
00:27:53,100 –> 00:27:54,750
I think that sends you
into a death spiral,

492
00:27:55,170 –> 00:27:58,020
but we might have to get pull
back and get more efficient.

493
00:27:58,020 –> 00:28:02,070
And so let’s talk about that
actually for a little bit.

494
00:28:02,070 –> 00:28:05,340
So where can you be led astray?

495
00:28:05,370 –> 00:28:07,650
I think you just made a post
on LinkedIn about this, right?

496
00:28:07,650 –> 00:28:12,120
Where you start looking at performance,
which feels like the smart thing to do,

497
00:28:12,330 –> 00:28:14,250
looking at ROAS and whatnot, and
you’re like, well, great, well,

498
00:28:14,250 –> 00:28:17,670
let’s just cut the lowest ROAS
campaigns and channels. We’ll be fine.

499
00:28:19,440 –> 00:28:21,480
How does that lead you astray?

500
00:28:21,480 –> 00:28:24,900
And if you want to talk about your
specific example to help illustrate these

501
00:28:24,900 –> 00:28:25,733
points, that’d be great.

502
00:28:26,220 –> 00:28:26,820
Yeah, totally.

503
00:28:26,820 –> 00:28:29,190
I think the other one you’re referring
to is I think branded search,

504
00:28:29,190 –> 00:28:31,830
which we were talking about
earlier. And I love using both a,

505
00:28:32,100 –> 00:28:35,940
because it can be really, if a brand
is spending a lot of money there,

506
00:28:35,940 –> 00:28:39,270
it can be a really great place to go
find those savings without impacting top

507
00:28:39,270 –> 00:28:42,540
line. But also frankly, it’s
really easy to understand.

508
00:28:42,840 –> 00:28:47,640
I think most people understand that
up and down the organizational chart

509
00:28:48,360 –> 00:28:51,870
across departments, everybody sort
of understands the idea of, Hey,

510
00:28:51,870 –> 00:28:53,640
if somebody’s already
searching for my brand,

511
00:28:53,970 –> 00:28:56,940
do I need to pay to get that
click and that conversion?

512
00:28:56,940 –> 00:29:01,650
And I found that just the fact that
it’s easy to understand can be a

513
00:29:01,650 –> 00:29:06,060
really good gateway to incrementality
testing because it’s easy to get buy-in.

514
00:29:06,060 –> 00:29:07,590
Everybody understands that idea,

515
00:29:07,740 –> 00:29:12,720
whereas it may be more challenging
to express that idea in

516
00:29:12,720 –> 00:29:15,870
other types of campaigns.
But branded search is a good example,

517
00:29:16,050 –> 00:29:19,110
and the example that you’re referring to,

518
00:29:20,040 –> 00:29:22,950
kind of a midsize brand that I was
working with went through that exact

519
00:29:22,950 –> 00:29:24,270
exercise, had to cut budgets.

520
00:29:26,190 –> 00:29:30,480
They looked at up and down the campaigns
they were running. It was like, Hey,

521
00:29:30,480 –> 00:29:33,600
we just got to make the best decision
we can with the best available data.

522
00:29:34,620 –> 00:29:39,210
They were basically running p max
non-branded search and branded search and

523
00:29:39,720 –> 00:29:44,050
p max and branded search where had
the best attributed roas Best CPA

524
00:29:44,950 –> 00:29:49,420
non-brand was really hard to justify in
a lower budget kind of environment based

525
00:29:49,420 –> 00:29:53,770
off the attribution data cut that leaned
a little bit more into branded search

526
00:29:53,770 –> 00:29:57,910
as a percentage of their budget.
And over the next couple months,

527
00:29:58,600 –> 00:30:03,550
new customers in total revenue
was declining despite the

528
00:30:03,550 –> 00:30:06,820
attributed ROAS and CPA
looking even better than ever.

529
00:30:07,900 –> 00:30:12,010
And that’s where was brought
in, looked at all these things,

530
00:30:12,010 –> 00:30:16,870
saw the loose correlation to
non-brand and new customer

531
00:30:16,870 –> 00:30:18,100
acquisition and top line,

532
00:30:18,760 –> 00:30:21,670
just the general skepticism that
many have around branded search,

533
00:30:21,670 –> 00:30:24,160
especially in a low
competition environment,

534
00:30:24,160 –> 00:30:29,110
which they were in. There weren’t many
competitors in the auction that we

535
00:30:29,110 –> 00:30:32,140
could see in Auction Insights. So yeah,

536
00:30:32,140 –> 00:30:35,380
ran a very blunt instrument
match market test,

537
00:30:35,380 –> 00:30:40,360
which at a brand of that size and for a
branded search I don’t think is ever a

538
00:30:40,360 –> 00:30:43,990
bad idea. And yeah, no
impact to branded search.

539
00:30:43,990 –> 00:30:45,520
It was about 20% of their budget,

540
00:30:45,520 –> 00:30:49,090
which was substantial that you
can either make the decision,

541
00:30:49,090 –> 00:30:53,860
I’m going to put that 20% back in
my pocket or save it for a rainy day

542
00:30:53,860 –> 00:30:58,480
or give it to some other
place in the org or say, Hey,

543
00:30:58,480 –> 00:31:03,220
I’m going to redistribute this to
something that I see in correlation

544
00:31:03,220 –> 00:31:06,130
data that might help
drive top line backup.

545
00:31:06,130 –> 00:31:10,660
Let’s reinvest that in non-brand as
opposed to keeping it in branded. Again,

546
00:31:10,810 –> 00:31:13,450
complete opposite of what
attribution would say.

547
00:31:14,170 –> 00:31:17,380
And you see that a lot frankly with
branded search is an easy one to pick on.

548
00:31:17,800 –> 00:31:18,940
Same with retargeting,

549
00:31:20,110 –> 00:31:24,490
but really anything that’s especially
challenging with the black box

550
00:31:24,700 –> 00:31:25,780
solutions that blend,

551
00:31:25,780 –> 00:31:29,560
and I’m sure we could do a whole talk
show on p max Advantage plus some of the

552
00:31:29,560 –> 00:31:33,730
things that bundled together historically
radically different levels of

553
00:31:33,730 –> 00:31:36,850
incrementality can be a real challenge
when you’re then measuring on

554
00:31:36,850 –> 00:31:41,800
attribution. But yeah, a
ranty way of saying yes,

555
00:31:42,460 –> 00:31:47,440
finding areas to cut oftentimes
if you follow the attribution kind

556
00:31:47,440 –> 00:31:51,760
of data can lead to really kind
of impactful in a negative way

557
00:31:52,300 –> 00:31:56,440
business outcomes because the attribution
view just does not take into account

558
00:31:57,190 –> 00:32:00,610
what would’ve happened
without the presence of those
ads like Incre Ality does.

559
00:32:01,510 –> 00:32:05,110
And so can definitely lead brands
astray as they’re looking to cut.

560
00:32:05,770 –> 00:32:07,690
Yeah, really interesting. And yeah,

561
00:32:07,900 –> 00:32:11,830
max notorious for leaning into
remarketing or branded search.

562
00:32:11,830 –> 00:32:15,100
If you’re not diligent about that, it
can lean into both of those things.

563
00:32:15,100 –> 00:32:17,260
And so got to be mindful of that.

564
00:32:17,650 –> 00:32:20,080
You also quoted something
that totally ties into this.

565
00:32:20,320 –> 00:32:24,580
It’s from a shop talk talk that
you went to shop Talk the show,

566
00:32:24,880 –> 00:32:29,050
and I can’t remember who said
it, but if you see high roas,

567
00:32:29,800 –> 00:32:34,660
I know something is wrong and that the
auto targeting is just finding existing

568
00:32:34,810 –> 00:32:39,530
customers. Do you remember actually
who said that and unpack a little bit?

569
00:32:40,040 –> 00:32:45,020
Yeah, I forget his name and I could
look real quick. He worked for.

570
00:32:45,740 –> 00:32:46,573
Mic.

571
00:32:47,120 –> 00:32:51,350
The Post Dan Danone, the big CPG.

572
00:32:52,580 –> 00:32:54,140
Yeah, I just really appreciated
that quote because I

573
00:32:56,060 –> 00:33:00,530
mean always wonder if I live in sort of
a bubble of being super passionate about

574
00:33:00,770 –> 00:33:02,720
incrementality versus attributed metrics,

575
00:33:03,140 –> 00:33:06,620
but that was just really refreshing to
hear because I don’t think that’s the

576
00:33:06,620 –> 00:33:07,453
natural.

577
00:33:08,810 –> 00:33:09,643
It’s not.

578
00:33:10,190 –> 00:33:11,060
Thought in people’s.

579
00:33:11,060 –> 00:33:12,230
Head spend more.

580
00:33:13,160 –> 00:33:16,310
But I really think it should
kind of spark some skepticism,

581
00:33:16,310 –> 00:33:20,120
especially when your goal really
is to try to drive new customers.

582
00:33:22,040 –> 00:33:22,730
My first,

583
00:33:22,730 –> 00:33:27,440
especially if you think about both
incrementality in the context of a SC

584
00:33:27,440 –> 00:33:31,280
or pex that’s blending retargeting
and prospecting by default

585
00:33:32,600 –> 00:33:34,610
and knowing diminishing returns

586
00:33:36,290 –> 00:33:39,320
are my first dollars, yes, they’re
going to be the most effective,

587
00:33:39,320 –> 00:33:44,240
but if they are focused on people that
are already buying from me and my goal in

588
00:33:44,240 –> 00:33:45,290
my head is new customers,

589
00:33:45,440 –> 00:33:50,330
I should be shocked that I can
spend a hundred dollars and drive

590
00:33:51,110 –> 00:33:53,240
this amazing new customer revenue

591
00:33:55,100 –> 00:33:59,300
and not think that something is up or
even over time as I continue to spend

592
00:33:59,810 –> 00:34:03,380
our BS meters should probably
go up a little bit more.

593
00:34:04,100 –> 00:34:07,790
And I don’t think they do by default. So
I found that comment really refreshing.

594
00:34:09,080 –> 00:34:09,860
Yeah, I think that
really illustrates that,

595
00:34:09,860 –> 00:34:13,130
right where it’s like most of us would
think, oh, ROAS is going up great,

596
00:34:13,130 –> 00:34:14,120
we’re printing money.

597
00:34:14,540 –> 00:34:19,490
Whereas maybe you should say BS
detector, something’s wrong here.

598
00:34:19,490 –> 00:34:23,060
This campaigns leaning into customers
that we’re going to buy anyway.

599
00:34:23,150 –> 00:34:26,600
And I’ll give two examples here to
illustrate this a little bit more.

600
00:34:26,860 –> 00:34:29,480
And I’ll also, since we’ve been
picking on branded search so much,

601
00:34:29,480 –> 00:34:32,870
I’ll share a couple of ways I
think we should use it. One.

602
00:34:33,380 –> 00:34:33,710
If.

603
00:34:33,710 –> 00:34:35,570
Other competitors are
aggressively bidding on,

604
00:34:36,680 –> 00:34:40,460
just know that if you’re not Nike and
you’re not Adidas and you’re not like Ford

605
00:34:40,460 –> 00:34:44,300
or something, it’s not a
lock. If it’s a new customer,

606
00:34:45,140 –> 00:34:46,850
they could be swayed by a competitor.

607
00:34:46,850 –> 00:34:49,760
And that’s generally how we
like to separate it out is like,

608
00:34:50,090 –> 00:34:54,530
let’s have branded search for returning
customers and let’s make that crazy

609
00:34:54,530 –> 00:34:56,570
efficient or just turn it off altogether.

610
00:34:56,750 –> 00:34:56,990
If.

611
00:34:56,990 –> 00:35:00,530
It’s a new customer, then again,
we want it to be very efficient,

612
00:35:00,530 –> 00:35:03,890
but maybe we want it on because we
don’t want our competitor to come in and

613
00:35:03,890 –> 00:35:08,690
swipe us to give and swipe our
customer. And so one example of this,

614
00:35:08,690 –> 00:35:12,920
I did a podcast with Brian Porter,
he’s the co-founder of Simple, modern,

615
00:35:12,920 –> 00:35:17,690
great Drinkware brand has become a friend
and they did a study incrementality

616
00:35:17,690 –> 00:35:19,640
study and they found, I’ll
get these numbers off,

617
00:35:19,640 –> 00:35:23,150
but it was like branded
search was 10% incremental.

618
00:35:23,510 –> 00:35:28,220
So basically what that means is if it
shows that I got a hundred new customers

619
00:35:28,220 –> 00:35:29,180
from Branded Search,

620
00:35:29,810 –> 00:35:33,080
I probably would’ve gotten 90 of
those if I had shut it off, right?

621
00:35:33,080 –> 00:35:35,000
Only 10% were incremental.

622
00:35:35,570 –> 00:35:39,720
So then what you would need to do there
is you need a 10 x row as on branded

623
00:35:39,720 –> 00:35:42,030
search for it to even make
sense. If it’s below that,

624
00:35:42,030 –> 00:35:45,840
you’re completely wasting
money. Pair that with,

625
00:35:45,840 –> 00:35:50,130
and you and I were commenting
on the House analytics, HAUS,

626
00:35:50,130 –> 00:35:54,750
Olivia Corey and team did 190
incrementality studies involving

627
00:35:54,780 –> 00:35:59,430
YouTube and they showed with
tremendous amounts of rigor

628
00:35:59,430 –> 00:36:00,660
that hey,

629
00:36:00,660 –> 00:36:05,220
YouTube is probably 342 times more

630
00:36:05,220 –> 00:36:08,340
incremental, meaning if
you see a one in platform,

631
00:36:08,340 –> 00:36:11,880
it’s actually like a 3 42 in
terms of incremental impact.

632
00:36:11,880 –> 00:36:16,170
And so wildly different
between those two. But again,

633
00:36:16,740 –> 00:36:19,800
we’re just so drawn to in platform
row as man, we’ll just say spin,

634
00:36:19,800 –> 00:36:23,280
spin spend on p max and branded search
when really we should be saying,

635
00:36:23,820 –> 00:36:27,270
let me lean into YouTube or let
me lean into top of funnel meta.

636
00:36:27,870 –> 00:36:31,350
I think both those examples
too are really good examples.

637
00:36:31,560 –> 00:36:33,480
To me it also speaks
though to the importance of

638
00:36:35,070 –> 00:36:39,540
cost per incremental almost being
more important than incremental

639
00:36:40,290 –> 00:36:43,200
percent incremental. And that’s something
I always use with branded search.

640
00:36:43,200 –> 00:36:45,750
I think you and I have a very similar
feeling around branded search.

641
00:36:46,590 –> 00:36:48,090
There’s definitely a
time and a place for it,

642
00:36:48,090 –> 00:36:51,420
and it’s one of those things where
it might not matter that it’s 10%

643
00:36:51,420 –> 00:36:55,680
incremental, 10% incremental relative
to what Google’s attributing.

644
00:36:57,230 –> 00:37:01,350
If your attributed CPA
is a dollar and now it’s

645
00:37:01,860 –> 00:37:02,580
$10,

646
00:37:02,580 –> 00:37:07,290
but your margin when you sell a
product is a thousand dollars like

647
00:37:07,860 –> 00:37:08,880
hammer that all day long,

648
00:37:09,210 –> 00:37:12,420
that cost per incremental is still
extremely profitable and valuable.

649
00:37:13,560 –> 00:37:15,000
And same with the YouTube piece.

650
00:37:15,570 –> 00:37:20,310
If YouTube was four times as
incremental as Google said,

651
00:37:20,580 –> 00:37:22,440
but your YouTube was crazy expensive,

652
00:37:22,590 –> 00:37:24,780
it still might not be worth it
even though it’s four times.

653
00:37:24,780 –> 00:37:25,500
More.

654
00:37:25,500 –> 00:37:27,900
Incremental than the platform was making.

655
00:37:27,900 –> 00:37:32,880
And that’s how I think a lot
about this with connected tv where

656
00:37:34,800 –> 00:37:39,660
connected TV can be super powerful
and maybe more so than linear tv,

657
00:37:39,840 –> 00:37:44,730
but if you can buy scatter
linear TV for a 10th

658
00:37:44,730 –> 00:37:46,800
of the cost of CTV,

659
00:37:47,250 –> 00:37:51,540
well it just has to be more
than a 10th as effective and

660
00:37:52,350 –> 00:37:54,180
it’s accreted, it’s a positive.

661
00:37:54,180 –> 00:37:58,890
So it becomes more of comparison
of a cost per than just a

662
00:37:58,890 –> 00:37:59,280
blanket.

663
00:37:59,280 –> 00:38:03,780
How incremental is something which I
always think is important to focus on and

664
00:38:03,780 –> 00:38:04,110
call out.

665
00:38:04,110 –> 00:38:05,460
To. Yeah, it’s so good.

666
00:38:05,460 –> 00:38:09,870
I mean measuring something in terms of
percentages can provide insights and help

667
00:38:09,870 –> 00:38:13,020
make decisions, but ultimately
it’s the cost per right.

668
00:38:14,220 –> 00:38:17,460
Translate that into real dollars
to see if it makes sense.

669
00:38:17,940 –> 00:38:19,380
100% agree with you,

670
00:38:19,380 –> 00:38:22,560
but I think this also goes back
to and use your linear TV example,

671
00:38:22,560 –> 00:38:25,770
and I still love TV and
connected TV and stuff. Again,

672
00:38:25,770 –> 00:38:27,900
I’ll use YouTube just because
I’ve got the numbers in my brain,

673
00:38:27,900 –> 00:38:32,310
but with YouTube sometimes
we’ll see a $5 CPM or a

674
00:38:32,310 –> 00:38:36,370
$7 CPM in certain audiences
compared to other channels that are

675
00:38:36,370 –> 00:38:39,940
15, 20, 30, 50, whatever.
Totally. And I’m like, well,

676
00:38:39,940 –> 00:38:44,680
if we’re reaching the right person
and if the message and offer are

677
00:38:44,680 –> 00:38:49,270
good, how could this not work? And it’s
one of those things where it’s like,

678
00:38:49,270 –> 00:38:52,030
okay, we’re either one of those is
off, we’re talking to the wrong person,

679
00:38:52,030 –> 00:38:52,960
that’s the wrong message,

680
00:38:53,170 –> 00:38:56,320
or we’re just not measuring it properly
and that’s where we need to look at it.

681
00:38:56,350 –> 00:38:58,690
So did you have a thought on that?

682
00:38:58,960 –> 00:39:00,490
You another question on
MM here in just a second.

683
00:39:00,700 –> 00:39:03,520
Yeah, yeah, totally. But it
made me think of the idea of,

684
00:39:05,200 –> 00:39:08,710
I think the reason I’m starting to become
way more bullish on any channel that’s

685
00:39:08,710 –> 00:39:11,620
historically been hard to measure
where I think there’s that arbitrage

686
00:39:11,620 –> 00:39:15,970
opportunity of costs are still relatively
low because people haven’t all moved

687
00:39:15,970 –> 00:39:17,590
in because it’s easy to attribute.

688
00:39:17,980 –> 00:39:20,800
It’ll be really interesting
with a house example,

689
00:39:20,950 –> 00:39:23,110
does that inspire a lot
more YouTube buyers?

690
00:39:23,470 –> 00:39:26,710
That’s something that Google
should have put out way long ago,

691
00:39:26,710 –> 00:39:30,940
but I think it would undermine
undermine search and that’s their bigger

692
00:39:30,940 –> 00:39:34,330
business. And I could do a whole
kind of rant and I’ll save you that,

693
00:39:34,330 –> 00:39:38,410
but the idea of incrementality first
measurement probably wouldn’t be great for

694
00:39:38,410 –> 00:39:40,360
the search business. So probably exactly,

695
00:39:40,360 –> 00:39:43,660
haven’t been able to make such a
good point that case on YouTube.

696
00:39:44,020 –> 00:39:46,750
But you think about all the channels
that have historically been harder to

697
00:39:46,750 –> 00:39:47,583
attribute,

698
00:39:47,830 –> 00:39:51,700
that’s where costs are deflated just
from a supply and demand perspective.

699
00:39:51,700 –> 00:39:56,560
So when you can move in and get CPMs at
five to $7 and it’s really effective,

700
00:39:56,560 –> 00:39:59,260
but most people that are measuring
through attribution don’t know it’s really

701
00:39:59,260 –> 00:40:04,210
effective, that’s a huge win for certain
period of time until everybody’s flood,

702
00:40:04,210 –> 00:40:05,290
everybody and the costs go.

703
00:40:05,290 –> 00:40:06,123
Up the market.

704
00:40:06,760 –> 00:40:10,270
I’m sure there’s a lot of people that
were not excited to see that study from

705
00:40:10,270 –> 00:40:14,770
house like dang it, that means my costs
are going up. I don’t like that at all.

706
00:40:14,890 –> 00:40:16,540
So really good man.

707
00:40:16,540 –> 00:40:19,900
So we talked about incrementality testing
and I think you can use tools like

708
00:40:19,900 –> 00:40:21,100
House and then there are others.

709
00:40:21,100 –> 00:40:24,130
We’re just talking about work magic and
there’s a number of others you can lean

710
00:40:24,130 –> 00:40:27,310
into. Full disclosure,
they’re pretty expensive,

711
00:40:27,880 –> 00:40:29,860
but you can also do stuff on your own too.

712
00:40:30,490 –> 00:40:32,320
If you’ve got someone that
can measure this stuff,

713
00:40:32,320 –> 00:40:36,700
you can do a little bit of it on your
own. What about the MMM side of things?

714
00:40:36,880 –> 00:40:41,770
What’s kind of the easy way to start
there? Is there an easy way to start?

715
00:40:42,640 –> 00:40:43,750
What do you recommend to people.

716
00:40:43,750 –> 00:40:48,730
There? I don’t know. I dunno if
there’s an easy way to do anything.

717
00:40:48,730 –> 00:40:52,870
I think, well, I guess
that’s not totally true.

718
00:40:52,870 –> 00:40:55,900
I think there’s some ways to
run relatively easy incre tests.

719
00:40:56,470 –> 00:40:58,270
So I think that’s the
easier place to start.

720
00:40:58,690 –> 00:41:01,570
Certainly you can always
ratchet up the scientific rigor.

721
00:41:01,570 –> 00:41:05,650
I think the problem with looking
for an easy MM solution is

722
00:41:06,940 –> 00:41:11,380
anybody could run a model with Robin or
there’s a lot of open source packages,

723
00:41:11,380 –> 00:41:13,180
but just because you can run a model,

724
00:41:16,150 –> 00:41:16,983
it could say anything.

725
00:41:18,400 –> 00:41:22,810
It’s not necessarily rooted in this
can all of a sudden predict the future

726
00:41:23,620 –> 00:41:26,110
and tell you exactly the
contribution from media.

727
00:41:26,110 –> 00:41:28,660
Whereas incrementality can do
that a little more out of the box.

728
00:41:28,660 –> 00:41:30,730
You may have wildly wide
confidence intervals,

729
00:41:31,750 –> 00:41:36,350
but it answers the question.
It gives you the comparison.

730
00:41:36,800 –> 00:41:37,970
I didn’t do it in this market,

731
00:41:37,970 –> 00:41:40,940
I did it in this market.
What is the Delta Media mix modeling?

732
00:41:40,940 –> 00:41:44,120
You could build a model
to tell sort of any story.

733
00:41:45,110 –> 00:41:48,560
The proof is sort of in the pudding of
if I do the thing that the model says,

734
00:41:48,980 –> 00:41:51,830
does it change my top line?

735
00:41:52,010 –> 00:41:55,340
Can I see over time that
when I listen to the model

736
00:41:57,140 –> 00:41:58,370
that improves my top line?

737
00:41:58,370 –> 00:42:03,140
So it’s a lot easier to get started
with incrementality testing.

738
00:42:03,140 –> 00:42:08,060
You can run poor man’s match
market tests as I sort you can just

739
00:42:08,060 –> 00:42:08,750
sort of pick,

740
00:42:08,750 –> 00:42:12,200
some markets historically behave
similarly and there’s certainly some risk

741
00:42:12,200 –> 00:42:15,710
there, but with a model you might
think that it’s an amazing model.

742
00:42:16,760 –> 00:42:21,230
I just don’t feel like there’s a great
place to DIY that together without some

743
00:42:21,230 –> 00:42:25,220
real scientific or statistical
rigor. Or if you do,

744
00:42:25,220 –> 00:42:29,900
you’ve just got to try to prove it over
and over by taking some big swings. And

745
00:42:29,900 –> 00:42:30,733
that’s really,

746
00:42:31,430 –> 00:42:35,120
I sort of feel like you can get away
with the kind of feel it sort of tests

747
00:42:35,750 –> 00:42:38,630
without really running a true
incrementality test or model.

748
00:42:38,990 –> 00:42:42,290
If you’re a small enough business and
you spend a decent amount on Facebook,

749
00:42:42,980 –> 00:42:44,780
maybe you’re not willing
to turn off Facebook,

750
00:42:44,780 –> 00:42:48,080
but are you willing to drastically
increase spend and see if you can feel

751
00:42:48,080 –> 00:42:51,530
something at the top line? Okay, then
what happens if you cut it in half?

752
00:42:51,530 –> 00:42:52,070
What happens?

753
00:42:52,070 –> 00:42:56,840
And start to understand those curves on
your own is probably a less risky way

754
00:42:56,840 –> 00:43:01,430
than trying to, I’ve never done
anything in R and I’m going to run

755
00:43:02,480 –> 00:43:05,090
or done any sort of medium amount.
I’m going to try to run one.

756
00:43:05,090 –> 00:43:06,500
That’s probably a risky proposition.

757
00:43:06,680 –> 00:43:09,980
Yeah, it’s a really good insight. I’m
glad you answered the question that way.

758
00:43:09,980 –> 00:43:10,850
I think, yeah,

759
00:43:10,850 –> 00:43:15,020
leaning into the poor man’s incrementality
test or just leaning really heavily

760
00:43:15,020 –> 00:43:18,800
into a channel and measuring your top
line if you’ve got a small enough business

761
00:43:18,800 –> 00:43:22,280
to look at that, but probably if
you’re going to lean into MM M1,

762
00:43:22,280 –> 00:43:26,360
you need a couple years of data and so
to be able to make some correlations and

763
00:43:26,360 –> 00:43:31,280
you probably need to lean in to
someone or a tool with quite a bit of

764
00:43:31,280 –> 00:43:32,750
experience because you can do that astray.

765
00:43:33,080 –> 00:43:35,270
And on your comment on cost too.

766
00:43:35,270 –> 00:43:39,710
I mean it’s all relative and a lot of
times where you’re going to need a medium

767
00:43:39,710 –> 00:43:42,710
mix modeling is when you’re spending
a significant amount in a significant

768
00:43:42,710 –> 00:43:43,430
number of channels,

769
00:43:43,430 –> 00:43:46,550
which you’re probably only doing
if you are spending a lot total,

770
00:43:46,550 –> 00:43:50,060
which you’re probably only doing if your
revenue can support that high level of

771
00:43:50,060 –> 00:43:50,390
spend,

772
00:43:50,390 –> 00:43:55,190
which means that a tool may not be
all that expensive relative to the

773
00:43:55,190 –> 00:43:58,850
opportunity you could derive from
it, which is where I always net out.

774
00:43:59,600 –> 00:44:01,790
So I’m paying 10 or 20
grand for a tool monthly,

775
00:44:01,790 –> 00:44:05,750
but it’s allowing me to
redeploy millions in ad spend.

776
00:44:05,750 –> 00:44:09,860
And it totally in completely
makes sense. So Tom,

777
00:44:09,860 –> 00:44:12,080
this has been fantastic.
I’m just watching the clock.

778
00:44:12,080 –> 00:44:15,530
I know we’re kind of coming
up against it, but one,

779
00:44:15,530 –> 00:44:18,530
I recommend people follow you on LinkedIn.
You put out some awesome content.

780
00:44:18,530 –> 00:44:19,400
I love reading it.

781
00:44:19,700 –> 00:44:19,700
Thank.

782
00:44:19,700 –> 00:44:23,540
You. People should definitely follow
you on LinkedIn and you are, is it Tom,

783
00:44:23,840 –> 00:44:27,980
what is your handle on LinkedIn?
You are Thomas B. Leonard.

784
00:44:28,190 –> 00:44:29,990
Thomas B. Leonard. That’s
probably confusing.

785
00:44:30,200 –> 00:44:33,300
I’m very self-conscious of LinkedIn, so
I’m glad to thank you for saying that.

786
00:44:34,530 –> 00:44:36,120
I think it’s good, man. I think it’s
really good. I like it a lot. Yeah.

787
00:44:36,420 –> 00:44:39,720
Yeah, it’s been fun to start
doing connecting with folks.

788
00:44:40,710 –> 00:44:45,480
Definitely an area that had a lot
of excitement and passion for,

789
00:44:45,480 –> 00:44:47,610
it’s fun to have these
sort of conversations,

790
00:44:47,610 –> 00:44:51,420
so I appreciate you reaching out a
while ago and that we could connect.

791
00:44:51,420 –> 00:44:52,253
Absolutely.

792
00:44:52,380 –> 00:44:55,080
Man. Absolutely. So then if
other people were like, Hey,

793
00:44:55,080 –> 00:44:58,890
I just want to talk to Tom because maybe
you can help my brand or my business,

794
00:44:59,550 –> 00:45:04,110
how can they connect with you and who are
you looking to or who do you feel like

795
00:45:04,110 –> 00:45:04,943
you can help?

796
00:45:05,310 –> 00:45:09,210
Yeah, definitely appreciate that.
Yeah, reach out on LinkedIn.

797
00:45:10,890 –> 00:45:14,640
I spend time there. I love reading
everybody’s thoughts and content. So yeah,

798
00:45:14,640 –> 00:45:19,590
reach out on LinkedIn mostly we work
with consumer facing brands that

799
00:45:19,590 –> 00:45:24,300
are trying to understand where to
put the next dollar or where to pull

800
00:45:24,840 –> 00:45:29,520
in the scenarios. They have to really
kind of rescue people from attribution,

801
00:45:30,480 –> 00:45:35,310
trying to better understand where they
can get more with their ad dollars.

802
00:45:35,310 –> 00:45:38,520
I think to your point that you teed
up now is such an interesting time or

803
00:45:38,520 –> 00:45:40,890
anytime that there’s margin pressure,

804
00:45:42,180 –> 00:45:44,280
there’s more scrutiny
on a marketing budget.

805
00:45:45,090 –> 00:45:49,350
Really want to try to help
empower marketing teams to
feel more confident with

806
00:45:49,350 –> 00:45:52,800
what they’re doing and ultimately the
finance teams to feel more confident with

807
00:45:52,800 –> 00:45:57,330
what marketing team is doing. Hundred
percent. That’s where I love to plug in,

808
00:45:57,750 –> 00:46:01,440
but also just love to talk about this
stuff probably more than I should.

809
00:46:01,440 –> 00:46:03,150
So always open to the conversation.

810
00:46:04,050 –> 00:46:05,730
Yeah, I talk about that a lot.

811
00:46:05,730 –> 00:46:10,620
I’ve read analytics and measurement
books on vacation and my wife

812
00:46:10,620 –> 00:46:14,400
is like, what is wrong with you? And I’m
like, it’s interesting. I don’t know.

813
00:46:14,400 –> 00:46:18,960
I like it. And so totally, we are
just a different breed I suppose,

814
00:46:18,960 –> 00:46:20,640
but I love that.

815
00:46:20,640 –> 00:46:24,630
And then I think this is a great way to
end it where if I’ve got an extra dollar

816
00:46:24,900 –> 00:46:29,490
to spend on marketing, where do I put
it? If I need to cut a dollar of spend,

817
00:46:29,550 –> 00:46:30,930
where do I cut it from?

818
00:46:31,260 –> 00:46:36,060
And that’s really what
this approach is about MMM

819
00:46:36,120 –> 00:46:39,570
and incrementality. And so
I think their necessities,

820
00:46:39,570 –> 00:46:43,620
I think attribution is broken and or
misleading in so many different ways.

821
00:46:44,550 –> 00:46:48,360
There’s some correlations there, so we
don’t have to throw it out completely,

822
00:46:48,690 –> 00:46:51,840
but I do believe you need to lean
into MMM and incrementality for short.

823
00:46:51,840 –> 00:46:56,670
So connect with Tom on LinkedIn.
And with that, we’ll wrap.

824
00:46:56,670 –> 00:47:01,500
Tom’s been fantastic. Thanks for the
time, the insights and the energy. Yeah.

825
00:47:02,340 –> 00:47:04,590
Thanks so much Brett
time. Glad to connect.

826
00:47:05,040 –> 00:47:09,900
Absolutely. And as always, thank you for
tuning in. We’d love to hear from you.

827
00:47:09,900 –> 00:47:12,060
If you found this episode helpful,

828
00:47:12,660 –> 00:47:15,690
someone else in the D two C space or
marketing space, and you think, man,

829
00:47:15,690 –> 00:47:18,570
they got to listen to this, please
share it. We mean the world to me.

830
00:47:19,020 –> 00:47:21,510
And with that, until next
time, thank you for listening.