This is a review of the literature of field experimental studies of markets. The main results covered by the review are as follows: (1) Generally speaking, markets organize the efficient exchange of commodities; (2) There are some behavioral anomalies that impede efficient exchange; (3) Many behavioral anomalies disappear when traders are experienced.
Economists are increasingly turning to the experimental method as a means to estimate causal effects. By using randomization to identify key treatment effects, theories previously viewed as untestable are now scrutinized, efficacy of public policies are now more easily verified, and stakeholders can swiftly add empirical evidence to aid their decision-making. This study provides an overview of experimental methods in economics, with a special focus on developing an economic theory of generalizability. Given that field experiments are in their infancy, our secondary focus pertains to a discussion of the various parameters that they identify, and how they add to scientific knowledge. We conclude that until we conduct more field experiments that build a bridge between the lab and the naturally-occurring settings of interest we cannot begin to make strong conclusions empirically on the crucial question of generalizability from the lab to the field.
The traditional approach in experimental economics is to use a between-subject design: the analyst places each unit in treatment or control simultaneously and recovers outcome differences via differencing conditional expectations. Within-subject designs represent a significant departure from this method, as the same unit is observed in both treatment and control conditions sequentially. While some might consider the design choice straightforward (always opt for a between-subject design), I contend that researchers should meticulously weigh the advantages and disadvantages of each design. In doing so, I propose a categorization for within-subject designs based on the plausibility of recovering an internally valid estimate. In one instance, which I denote as stealth designs, the analyst should unequivocally choose a within-subject design rather than a between-subject design.
Excellence gaps - disparities in advanced academic achievement - between racial groups appear by age 8 or 9 and persist throughout secondary school in the United States. About one-third of the gap is due to socio-economic status and one-tenth to school factors, indicating that policies should address both educational and local environments.
To identify effective policies that can scale, a third option should be added to traditional A/B tests, that accounts for the realities of a programme implemented at scale. By flipping the traditional research and policy-development model, researchers can generate policy-based evidence to help policymakers scale the best policies.
This paper examines the impact of payment choice on charitable giving with a door-to-door fund-raising field experiment. Respondents can donate cash only, use debit only, or have both options. Cash donations have lower visibility vis-a-vis solicitors than debit card donations. When debit replaces cash, participation drops by 87 percent. Conditional on participation, donors in the Debit-only treatment give more than donors in Cash-only. In Cash&Debit, almost all donors prefer cash; participation decreases compared to Cash-only. Physical attractiveness of both female and male solicitors increases contributions. Solicitor self-confidence has a negative impact.
In 2019, I put together a summary of data from my field experiments website that pertained to natural field experiments (Harrison and List, 2004). Several people have asked me for updates. In this document I update all figures and numbers to show the details for 2024. I also include the description from the original paper below.
Uditi Karna, Min Sok Lee, John List, Andrew Simon, and Haruka Uchida
Educational disparities remain a key contributor to increasing social and wealth inequalities. To address this, researchers and policymakers have focused on average differences between racial groups or differences among students who are falling behind. This focus potentially leads to educational triage, diverting resources away from high-achieving students, including those from racial minorities. Here we focus on the "racial excellence gap" - the difference in the likelihood that students from racial minorities (Black and Hispanic) reach the highest levels of academic achievement compared with their non-minority (white and Asian) peers. There is a shortage of evidence that systematically measures the magnitude of the excellence gap and how it evolves. Using longitudinal, statewide, administrative data, we document eight facts regarding the excellence gap from third grade (typically ages 8-9) to high school (typically ages 14-18), link the stability of excellence gaps and student backgrounds, and assess the efficacy of public policies. We show that excellence gaps in maths and reading are evident by the third grade and grow slightly over time, especially for female students. About one third of the gap is explained by a student's socioeconomic status, and about one tenth is explained by the school environment. Top-achieving racial minority students are also less likely to persist in excellence as they progress through school. Moreover, state accountability policies that direct additional resources to reduce non-race-based inequality had minimal effects on the racial excellence gaps. Documenting these patterns is an important step towards eliminating excellence gaps and removing the "racial glass ceiling".
Given the replacement of cash with cell phone payments, people who are asked to donate to charity can easily promise a donation but delay the transfer until a later date. This may be a way to get out of the ask-situation with a positive image while maintaining the flexibility not to donate. This study explores whether charities can make people keep their promises by making such promises more explicit and more formal. In a door-to-door fund-raising field experiment, we vary the strength of the promise that donors make. Besides a control group where people can promise to donate, we apply two treatment groups. In the first treatment, donors are asked to verbally pledge a precise amount. In a second treatment, this amount is in addition put on paper with the solicitor's signature added. Both treatments are aimed at making it morally more expensive not to keep promises. Our results show that: (1) the majority of people do not follow through on their promise to donate; (2) donors who pledge an explicit amount more often keep their promise. The more formal the commitment, the closer the amount donated is to the amount promised; (3) many participants refuse to pledge a donation amount when asked, and those who refuse donate significantly less.
