How (Not) To Test Theory with Data: Illustrations from Walasek, Mullett, and Stewart

Abstract

André and de Langhe (2021) pointed out that Walasek and Stewart (2015) estimated loss aversion on different lotteries in different conditions. Because of this flaw in the experimental design, their results should not be taken as evidence that loss aversion can disappear and reverse, or that decision by sampling is the origin of loss aversion. In their response to André and de Langhe (2021); Walasek et al. (2021) defend the link between decision by sampling and loss aversion. We take their response as an opportunity to emphasize three guiding principles when testing theory with data: (a) Look for data that are uniquely predicted by the theory, (b) Do not ignore data that contradict the theory, and (c) If an experiment is flawed, fix it. In light of these principles, we do not believe that Walasek et al. (2021) provide new insights about the origin and stability of loss aversion.