 |
Forecast evaluation is a long-standing issue in applied econometrics. Standard tests suffer however from the presence of time-varying volatility in many applications. Besides heteroskedasticity, we tackle the important issues of time-variation in relative forecast ability and estimation uncertainty. To this end, we study the fluctuation test of Giacomini and Rossi (2010) and of two new CUSUM- and Cramer-von Mises based tests. While ``fixed-b'' arguments (Kiefer and
Vogelsang, 2005) provide refinements in the use of heteroskedasticity and autocorrelation consistent variance estimators, the resulting limiting distributions of test statistics depend on the unconditional variance changes over time for both small-$b$ and fixed-b approaches. To restore asymptotically pivotal inference, we employ a wild bootstrap approach. After establishing necessary theoretical results, simulations quantify the size distortions from using the original fixed-b approach and show the suggested bootstrap to work reliably. The empirical part studies the (time-varying) superiority of professional forecasters relative to naive no-change predictions in real-time. We exploit the most comprehensive database of the Survey of Professional Forecasters (SPF) and analyze forecasts for several key macroeconomic variables over a sample from 1969 to 2017. Our findings suggest that not accounting for heteroskedasticity seriously affects outcomes of tests for equal predictive ability and time-variation: wild bootstrap inference yields convincing evidence for the superiority of the SPF in most cases, while tests using asymptotic critical values provide remarkably less. Moreover, we find significant evidence for time-variation of relative predictive power; the dominance of the SPF appears to weaken considerably after the ``Great Moderation''.
