As a rule, econometric data are non-experimental. The same dataset is used to select the model and, conditional on the selected model, to forecast. In applied econometrics, however, one typically reports the properties of the (conditional) forecast, ignoring the fact that its properties are affected by the model selection (pretesting). This is wrong, and in this paper we show that the error can be substantial. We obtain explicit expressions for this error. To illustrate the theory we consider the regression approach of Pesaran and Timmermann (1994) to stock market forecasting, and show that their proposed recursive predictions are much less robust than naive econometrics might suggest.