Economic Modelling: Status Quo Ante

While there have been alternatives proposed (see for instance this post), large scale structural and stochastic models are still the bread and butter of macropolicy. Yet the inability of virtually every statistical model to provide substantive guidance on policy issues remains a problem.

This article on VoxEU provides an insight as to why (excerpt; emphasis added):

Dynamic stochastic general equilibrium models and their forecasts
Rochelle M Edge & Refet S. Gürkaynak

Dynamic stochastic general equilibrium (DSGE) models represent a major strand of the modern macroeconomics literature and are an important tool for policy analysis at central banks...

...The success of the DSGE model-based forecasts relative to other methods was viewed as evidence in favour of DSGE models’ reliably capturing the dynamics in the data…

...To see the absolute forecasting ability of the DSGE model, we run a series of standard forecast efficiency tests, where the realised inflation is regressed on forecasts made at different times in the past. A good forecast should have a zero intercept and unit slope as well as a high R-squared. Table 1 shows the efficiency tests for DSGE model forecasts of inflation at different maturities and demonstrates clearly that the forecasts are very poor. R-squareds at all horizons are essentially zero, implying no forecasting ability. All Figure 1 is therefore telling us is that all other forecasting methods perform just as poorly....

…What do we learn from this? That over the post-1992 time-period, which we use to evaluate the DSGE model, inflation has been essentially unforecastable. (Other papers use similar sample periods.) This is in line with the finding of Stock and Watson (2007), who showed that inflation since the Great Moderation is characterised by a diminished persistent (forecastable) component and a larger transitory (unforecastable) component...

…But what does that say about the use of DSGE models in central banks; both as a tool for policy analysis as well as a tool for forecast generation? We argue that in both cases the answer is “nothing”. The finding that inflation is not forecastable over the Great Moderation period is consistent with the predictions of the DSGE model given the strong monetary policy rule estimated for this period. Specifically, since under this rule the policymaker will alter the interest rate to counter forecastable deviations of inflation from the target, the rule will eliminate forecastable movements in inflation and leave only unforecastable shocks to drive fluctuations. Thus, our finding of low forecast performance is not necessarily evidence against the validity of the model...

...The interesting question is whether the conditional forecasts of the DSGE model are sensible. That is, whether the model can reasonably accurately answer questions along the lines of “what would inflation do if the interest rates were kept constant for a year?” It is entirely possible that the model will predict the path of inflation under the counterfactual policy path quite well, while having a poor unconditional forecast record as it is the internal dynamics that imply unforecastable inflation.

Ok, what they’re arguing here is essentially a version of the Lucas Critique – since policy affects the variables of interest and the variables concurrently affect policy, then any structural relationships that policy actions leverage on will break down. In this case, forecasts of inflation (or economic growth) will not be realised if policymakers act, thus rendering those forecasts wrong.

It’s been the fashion to try and overcome this limitation that model makers have added micro-foundations to large scale models i.e. building from the ground up with equations detailing the reactions of individual agents to shocks in variables, thus providing some basis for endogenous change in structural relationships. Even with this innovation though (now in its third decade), its still clear that these new models don’t generate good forecasts, and haven’t fully captured the dynamics of endogenously generated policy shocks.

One strand of thought being actively investigated is dispensing with the “representative agent”, sometimes sarcastically referred to as homo economicus – the utility maximising individual or company operating under the assumption of full rational expectations. But the complexity of these alternative models precludes their widespread adoption.

So even as DSGE’s and their older cousins Computed General Equilibrium (CGE) models have had their limitations displayed, the point is taken that while forecast generation is likely to remain poor, there’s still a use for these models in answering hypothetical policy questions.

A related working paper covers the improvements to macro models, that while probably won’t add to forecasting ability, might be useful in monitoring systemic risk (excerpt):

International macro-finance
Anna Pavlova Roberto Rigobon

…Financial markets and their role in international risk sharing have inspired a vast body of theoretical literature. Over the past 40 years, international finance and economics has evolved into a vibrant field spreading from the basic international version of the capital asset pricing model to some of the most sophisticated dynamic stochastic general equilibrium models.

Curiously, however, the research effort in economics has evolved almost in parallel with that in finance. In economics, the main focus has been on real quantities and international relative prices such as consumption, investment, current account, terms of trade and exchange rates. Meanwhile, international portfolio choice and international equity markets have been largely overlooked…

…Both approaches have been very useful, but they cannot address many questions pertaining to portfolio problems and to the international equity markets. Finance, on the other hand, has focused more on cross-country portfolio allocations and asset prices. Terms of trade and hence exchange rates have been largely overlooked because the majority of the models featured a single-good framework, in which forces of arbitrage equate terms of trade to unity. Models with endogenous portfolio selection, equity prices and time-varying terms of trade and exchange rates in a single framework have been quite rare...

...Unfortunately, most of the existing international macro models are not well-suited for dealing with these issues because they are missing equity markets and portfolio choice. A new and rapidly growing strand of literature, commonly known as international macro-finance, is trying to fill this gap. This new generation of macro models provides a redefinition of the current account (adjusted for capital gains on equity holdings) and modifies the standard theories of the current account. More generally, this research programme focuses on the interaction between the financial sector and the real economy, and as such can address a wide spectrum of issues such as contagion, composition of international portfolios, valuation effects, and others...

...Although we have learned a great deal from this strand of research, many questions remain open. In order to tackle more ambitious questions raised by the data and current events, the existing models certainly require improvements along several dimensions.

This research agenda includes market incompleteness, the addition of factor markets, inclusion of the financial sector in large scale models, and use of more complex estimation techniques. In other words, the watchword here is increased complexity of existing modelling frameworks, which in all honesty are pretty complex already.

So what’s the best way forward? The economics profession is beginning to get a handle on how best to address the policy and dogmatic failures in the runup to the Great Global Recession. Which way it will fall I honestly don’t know, but this is the biggest sea change in economic orthodoxy for nearly two generations, and I’m happy having a ringside seat.

Technical Notes:

1. Edge, Rochelle M & Refet S. Gürkaynak, "Dynamic stochastic general equilibrium models and their forecasts", VoxEU, February 2011
2. Edge, Rochelle M & Refet S. Gürkaynak, "How Useful are Estimated DSGE Model Forecasts?", The Federal Reserve Board Finance and Economics Discussion Series: 2011-11, January 2011
3. Pavlova, Anna & Roberto Rigobon, "International macro-finance", VoxEU, February 2011
4. Pavlova, Anna & Roberto Rigobon, "International macro-finance", NBER Working Paper No. 16630, December 2010