Quoting David Beckworth quoting Barry Ritholtz (excerpt):
Why Counterfactual Thinking Is Important
I recently made the case that many observers are not thinking properly about the Fed's Quantitative Easing (QE) programs. Using the analogy of George Bailey's life in the film It's a Wonderful Life, I argued that the critics who question the efficacy of the QE programs are doing the wrong counterfactual. Today, Barry Ritholtz makes the same point:
One of the analytical errors I seem to constantly come across is what I call the non-result result. It goes something like this:
If you do X, and there is no measurable change, X is therefore ineffective.The problem with this analysis is the lack of a control group, If you are testing a new medication to reduce tumors, you want to see what happened to the group that did not get the tested therapy. Perhaps their tumors grew and metastasized. Hence, no increase in tumor mass or spreading is considered a very positive outcome.
This seems to get loss in the debate over QE. The debate — either ignorantly or disingenuously — makes claims such as “Look how few jobs have been created, and look how high unemployment is.”
Understanding this logic, and lacking a control group, we must employ a counter-factual. The question one should be asking is “How many less jobs would have been created?; How much higher would unemployment be?”
I made the exact same point a year and a half ago:
Evaluating Public Policy: Dreaming of Parallel Earths
...Lots of things can have an impact on any given target variable, and you can’t assume that whatever changes occurred are purely from the effect of a particular policy. Improvements or a deterioration in the situation targeted by that policy could have come about from completely unrelated sources...
…In a nutshell, figuring out whether a particular public policy is working you’ll need to measure it, not against what it has achieved or not achieved, but against what would have happened in its absence…
...That’s one big reason why economists rely on models, which are simplified enough to be understandable, yet close enough to the real world to hopefully gain real insights. It’s also why the field of economics is increasingly mathematically demanding...
...But the underlying problem still remains – you can’t really rigorously evaluate the impact of any particular policy or set of policies based on what has happened alone. It has to be measured against the “would have been” and “might have been”.
This isn’t just about QE or the ETP or any other public policy, it’s about all of them. Next time somebody says a particular policy has succeeded or failed, ask what the counterfactual would be.
Posts
This "counter factual thinking" is a joke.
ReplyDeleteImagine today we don't have any QE but other policies that is exactly opposite of QE. Since Keynesians believe that without QE, everything will be worse so the *likely* is that we also didn't create much jobs and the economy is bad.
These policy makers can say the same thing - you need "counter factual thinking":-
“How many less jobs would have been created?; How much higher would unemployment be?” if we have done QE... [I copy and paste exactly the same questions and it works disregard to what policies you implemented, QE or not QE as long as the outcome is bad]
This works in personal life as "positive thinking". If we lost a leg, we told our self luckily we didn't lost 2 legs. If we don't have a car but just a motorbike, we tell our self luckily we don't need to ride a bicycle.
But for policy makers, this is lame excuse. Same thing when our government say our internet speed is faster than Nigeria and that our crime rates is lower than Iraq.
This brilliant "counter factual thinking" means nothing is measurable and nothing should be measured because anything can be compared to something worse that could have happened and all economic policies can be asked with the same 2 questions and make them great.
“How many less jobs would have been created?; How much higher would unemployment be?”
Try it yourself!
@anon 6.44
DeleteActually, counterfactuals is pretty much the whole basis of economic "science". Since we cannot conduct experiments, we build models instead. It's more about framing the proper research question, not a denial that things can be analysed.
Who said counterfactuals can't be measured?
Here's one example; here's another. To take a more recent and relevant question, how much of the recent spate in crimes is due to the release of detainees, and how much to due to the economy slowing down at the same time? If the data was public, this proposition would be amenable to econometric testing. And we can make an objective decision whether PCA is really necessary or not.
Also:
Since Keynesians believe that without QE, everything will be worse so the *likely* is that we also didn't create much jobs and the economy is bad.
This is a strange statement. Keynesian believe that an economy at the zero interest rate lower bound is in a liquidity trap, which means monetary policy cannot be effective and fiscal policy must be used instead i.e. they don't believe QE works.
Without reference to counter-factual thinking, all bad decisions made during good times will look like good decisions since good times generate positive outcomes regardless of government incompetence, and all good decisions made during bad times will look like bad decisions because the government is ultimately trying to limit damage rather than generate positive outcomes. That's no way to make decisions about public policy.
DeleteThis of course assumes that the best effort approximations made by economists are at least marginally better than a random stab in the dark.
@anon 4.51
DeleteAbsolutely right, especially that last bit...
Actually this is a simple theory. Say, how much youre supposed to drink in a day, 7/8 litres of water, really plain water ? That's what the doctors say to maintain your good health. Do we really drink that much or do we consume tea, coffee or other drink as well ? Some of course, beers and liquors.
ReplyDeleteNow on a counterfactual, is this measureable ? Logically, yes but no body would give exact result. " agak-agak boleh lah, minum tetap minum. Ada yang tidak minum dari pagi sampai petang. Only later at night "
Policy or set of policies, just mere theory that act as a guide. Mostly based on case studies or set of examples, norms. Those policy makers, indeed !
Then, without warning, they changed everything !
With reference to your article's last paragraph, QE, ETP and any other public policy, the counterfactual thinking, finally accredited to ' hired foreign con- Sultan or Economic journos con - Sultan...' as Chedet wrote in his blog....