p.381

Subject Evaluation in Social Experiments

Tomas Philipson
Larry V. Hedges

Abstract

This paper analyzes how self-interested subjects, as opposed to altruistic investigators, evaluate treatments in social experiments. We argue that the attrition behavior of subjects reveals their evaluation of treatments, and we discuss the usefulness of using such data in performing subject-based evaluation. We examine the possible causes of disagreements between investigators and subjects in their evaluation of treatments, and empirically assess the extent to which they disagree. Given that disagreements between subject evaluation and evaluation done by investigators could be due to sampling error, this paper provides an empirical framework for estimating the systematic level of disagreement in the presence of such errors. We illustrate this framework by estimating the extent of disagreement in clinical trials, and we find substantial evidence of over-approval by investigators in about one-third of the trials analyzed.