Flexible Demand Estimation with Search Data

Abstract

We propose a model of consideration and choice that uses search data to flexibly estimate co-search patterns for every pair of products. Using simulations, we show that estimating co-search flexibly rather than modeling the drivers of consideration comes at a modest cost in precision but guards against misspecification. The method is therefore particularly valuable in online markets, where the drivers of consideration — such as webpage layout or product recommendations — are often unobserved. Relative to using purchase data alone, search data substantially improves the precision of cross-price elasticity estimates, because products that are searched together tend to be close substitutes and because browsing data is far more abundant than purchases. We apply our approach to data from an online retailer and find that most products have a small set of close substitutes and that accounting for substitution patterns when setting prices raises the retailer's average mark-up from 36.5% to 47.5% and increases its profits.