On this page 2 sections
Each proposition stands on its own and names the chapter where its evidence is set out.

1. The effect of an ad is a difference between two worlds, and a report shows only one of them. Credit is the share of a sale a report assigns to an ad. Cause is the share that would disappear without it. Every measurement method is a way of estimating the missing world, or of not trying. (Chapter 1)
2. Advertising effects are small compared with the noise in sales. Typical elasticities are around 0.1 to 0.2, and confidence intervals for the return on a single campaign are often more than 100 percentage points wide. A very large reported return is more likely to reflect selection than effect. (Chapter 1)
3. A reported conversion is a claim made under a definition. The event, the link to a person and the time window are all choices. When a platform changes them, the number changes although no customer did. (Chapter 2)
4. When platform claims add up to more than the store's revenue, their return figures cannot be compared. Each platform credits itself for shared sales, so its ROAS is computed on an overlapping and unknown share of the same revenue. (Chapter 2)
5. The seller of the ad also grades it. Platform numbers are platform statements: often the best data available, never independent evidence. (Chapter 2)
6. Attribution divides credit by a rule, and when channels interact there is no single natural split. Channels that reinforce each other have causal effects that sum to more than 100% of the sales they jointly produce; channels that compete for the same buyers sum to less. Every attribution model forces the sum to 100%; a causal split exists only under stated assumptions about how interactions are shared. (Chapter 3)
7. Observational estimates tend to overstate advertising effects, often several times over. The overstatement is largest where targeting is strongest, such as retargeting and brand search, and better data or better algorithms do not remove it. (Chapter 4)
8. Locating a change is not explaining it. Decomposing a change into volume, mix and rate shows where it happened. Only an intervention shows why. (Chapter 4)
9. Random assignment is the most direct route from credit to cause. Ghost ads and geo experiments make it affordable. A test result must be adjusted for movement in the control group and scaled to the whole business before it is compared with total spend. (Chapter 5)
10. Budget should follow the marginal return, not the average. Saturation lowers the margin below the average, delayed effects hide it from short windows, and synergy makes each channel's margin depend on the others. (Chapter 6)
11. When evidence is weak, keep the budget and test. Trusting every platform number and discounting every platform number are both shortcuts. The evidence on each channel decides, and a short test is often cheaper than the mistake it prevents. (Chapter 6)
12. Attribution gives direction, experiments give truth, models give allocation, and calibration connects them. An AI analyst can climb only as high as that evidence reaches. In our test, language models applied these distinctions when the evidence was visible; their weak point was arithmetic scale. (Chapters 7 and 8)
What carries over from large advertisers to a small online store?
[Our reading] Most of the evidence in this book comes from large advertisers: 25 experiments with major US retailers and brokerages (Lewis & Rao, 2015), 663 and 2,226 Meta experiments (Gordon et al., 2023; Gordon et al., 2026), eBay's national search advertising (Blake et al., 2015) and 288 consumer brands on US television (Shapiro et al., 2021). A small online store differs in size, in how well buyers already know it, and in the channels it uses. Three kinds of findings carry over to different degrees.
Mechanisms carry over. Targeting selects people who would have bought anyway, platforms credit themselves for shared sales, a geo test has to be adjusted for its control group and scaled to the whole business, and the average return differs from the marginal one. None of these depends on the advertiser's size. They follow from how ads are delivered and how reports are built, and a small store that buys from the same platforms inherits them.
Effect sizes do not carry over directly. The 5% median lift, the average elasticity of 0.12 from studies published between 1960 and 2008 (Sethuraman et al., 2011) and eBay's return of −63% describe their own samples. Shapiro's and Lodish's estimates come from consumer packaged goods and established products advertised on television, and most of eBay's search spending reached frequent eBay users whose purchases did not depend on the ads (Chapter 4). A store that few people know may reach more buyers who would not otherwise have found it, which would make its effects larger than eBay's. The sources used in this book do not measure whether that holds for small stores.
The noise problem gets worse. Lewis and Rao's experiments covered $2.8 million of advertising with major retailers, and their median interval was still more than 100 percentage points wide. A conclusive experiment can require more than 10 million person-weeks of data (Chapter 1), far beyond a small store's traffic. The practical response is to test larger changes, such as pausing a whole channel, to run tests for longer, and to judge the interval against break-even rather than against zero, as in the worked example in Chapter 5.
Open questions
The book's own limits also mark the questions the evidence has not yet answered.
- Long-term brand effects. How can the effect on memory, preference and pricing power that tests lasting days or weeks do not see be measured precisely enough to decide on? (Front matter; Chapter 1)
- Beyond the targeted audience. How can the value of reaching people outside the audience a platform has chosen be brought into the view of user-level experiments? (Front matter)
- Sharing interactions. When channels reinforce each other, which assumptions about sharing the interaction are defensible, and how can an experiment test them? (Chapter 3)
- Carrying an experiment over. How far does a result from one campaign, period and budget level carry over to other budgets, times and channels? (Chapter 5)
- Keeping the model honest. When a business cannot test every channel every month, which experiments should check a media mix model's assumptions, and how often? (Chapter 7)
- The AI analyst on real reports. Can language models make the same distinctions when the decisive evidence is buried among dozens of rows and several traps occur together? (Chapter 8)
- The accuracy of attribution. Has click-based attribution become less accurate over time? The book leaves this as a hypothesis, not a measured trend. (Appendix C)