I've meant to write a post about academic papers for some time and I received an email recently that gives me a great opportunity.
“I generally think highly of [Dr. Redacted’s] work. . . But I’ve been puzzled by why his results sometimes weren’t more thorough – they are sometimes like abstract points . . . They can’t be applied without significantly more refinement and sometimes can lead to a wrong conclusion.”Exactly.
But the problem, I believe, is not with the academic research – which is as it should be – but with our expectations and our over-reading of the conclusions. Sometimes, we just miss the point. Other times, we try too hard to convert theory into practical advice. And lastly, we're not the intended audience.
Let's look at those one point at a time.
It seems that many people unfamiliar with how academic research works are looking for a study that shows the one way to plan for retirement that is better than all previous recommendations and is the single best way to plan. As they say on police dramas, that ain't gonna happen.
Research is often an “experiment” that provides new information that we should consider and perhaps use to modify our previous beliefs to some extent. Or as someone said, “science is the process of continually improving your answer.” If you expect the next big academic paper to provide “the answer” that supersedes everything we used to think that we knew was correct, your expectations are too high. A good piece of research simply improves on the previous answer. It rarely replaces it.
So, is research sometimes "like abstract points"? Yes, often. One definition of abstract is “theoretical” and most academic research is theoretical, that is “concerned with or involving the theory of a subject or area of study rather than its practical application.”
Molding theory into practical application is the next challenge. Sustainable withdrawal rates research is, I think, an excellent example. Most studies use constant-dollar withdrawals, a spherical cow that is useful for research but problematic in practical application. No rational retiree is going to continue to spend the same amount every year once she observes that she is going broke. The technique is useful to understand the process of sequence of returns risk, but it is way too simple a model to predict outcomes for an actual retiree.
As a highly-respected retirement researcher once told me, “Bengen did an outstanding job showing that sequence risk exists, but then trying to identify a safe withdrawal rate was a fool's errand.”
Unfortunately, Money magazine and other popular media outlets translated this research directly into practical advice for several years before the 2007-2009 market crash convinced them that people really need to spend less money when they become less wealthy. The risk is not so much that you will deplete your savings with SWR as that you will irretrievably lose your lifestyle.
The comment above that research papers can sometimes lead to wrong conclusions is spot on but, again, this is more a problem with our conclusions than with the research. Here's an example. Wade Pfau and Michael Kitces caused a stir (a sign of good research) by writing a paper entitled, Reducing Retirement Risk with a Rising Equity Glide Path. Their research suggests that “rising equity glide paths from conservative starting points can achieve superior results.” One well-known retirement expert immediately tweeted something to the effect of, “Great idea - let's put all 95-year olds 100% in stocks.”
That wasn't the point of the research, but the effects were predictable. Clients asked if I thought they should be following a rising glide path and invest more in equities when they are older. I told them that they should decide that when they are older – I have no idea what their financial situation will be when they are 95.
Wade Pfau agreed, telling me that a custom asset allocation will always be better. He believed their findings simply suggest that if we knew nothing about a client other than his or her age, or if the retiree were only willing to do the absolute minimum of planning, a rising glide path would be the best bet.
He added that this is probably the best strategy for a target-date mutual fund that needs to serve a broad range of retirement investing needs. He also felt that the most interesting finding of the research is that you can sometimes “reduce both the probability of failure and the magnitude of failure for client portfolios” by increasing market risk. Not that everyone should adopt a rising glide path upon retirement and stick with it come what may, though this is what many concluded.
Sometimes, we just miss the point. David Blanchett wrote a paper that inspired the term “the Blanchett Smile” (download PDF). Many readers understood his findings to suggest that expenses begin to decline at retirement, bottom out about the mid-point, and then increase until death, in the shape of a smile. The “smile” Blanchett found, however, showed the acceleration of spending, not the amount of spending. The amount always trended downward, at about 2% or so per year for appropriately-spending retirees, throughout retirement. Only the acceleration of the downward trend (it's first derivative) eventually turned upward.
Misreading academic papers isn't a problem only among do-it-yourself retirees. After I wrote about this at Advisor Perspectives, a top-notch retirement adviser privately thanked me. He said that he had been present when Blanchett presented the paper and that he completely missed this point, as did, he believed, everyone sitting near him.
My last point is an important one: we are not the intended audience for academic papers. They are written primarily for other academics, who will review the papers before publication. Consequently, they tend to make modest claims that can be strongly supported by the evidence provided. Claims more like “increasing portfolio risk has the potential to actually reduce both the probability of failure and the magnitude of failure for client portfolios” rather than “everyone should follow a rising glide path.”
(Be even more cautious of "pseudo-academic papers" that are not peer-reviewed and thoroughly cited.)
I'm not suggesting that you stop reading academic papers on retirement finance. I am suggesting that you understand their intent and use considerable care in trying to apply their findings to your own personal situation. Treat them as another piece of evidence and weigh them accordingly.
(Interestingly, I've found that the retirement field is not the only one where academic research has a strong following among lay readers. I found a website where non-scientists express great interest in cosmic physics. I find that encouraging.)
While academic papers aren't intended for a broad audience, the authors of those papers often write books, blogs and columns for the press that are. Wade Pfau, for example, does an excellent job of explaining his research to a broader audience of advisers and even do-it-yourselfers at several places, including newsletters to which you can subscribe, the Advisor Perspectives website, and his own Retirement Researcher blog. William Bernstein has written many brilliant books and over the years has intentionally made them more and more accessible.
Dr. Moshe Milevsky is another excellent writer. While I don't recommend most people tackle his papers about lifetime probability of ruin and the reciprocal gamma distribution (it kept me up nights until my tenth reading), I just finished the second edition of Pensionize Your Nest Egg, co-written by Milevsky and certified financial planner, Alexandra Macqueen, and find it quite readable. (Macqueen clearly helps balance Milevsky's inner quant.) It provides a strong argument for the circumstances under which and to what extent one should employ annuities. Spoiler: if you have a lot of wealth or a little, they may be less effective than for those in between.
Thank you for the well written article. I also recently finished reading "Pensionize Your Nest Egg" and agree that it provides excellent advice for a great many "middle class" retirees. However, many or even most financial planners are not advising these middle class clients to annuitize, delay SS, and to safely spend down their nest eggs even though it is probably their fiduciary duty to do so. Due to the way these advisers are paid, they have some huge and obvious conflicts to deal with. Perhaps a topic for an article someday?
ReplyDeleteI think there are a number of reasons an advisor might not recommend annuities. One, as you mention, is that some advisors are not compensated for annuity sales but are compensated for selling alternatives. Advisers have to make a living, so it is important to understand how they do so.
DeleteAnother reason they might not recommend annuities is that they genuinely believe that annuities have serious flaws. (They do, but every retirement income solution has some flaw.)
A third possible reason is that they believe annuities are a good idea for many situations, but not in your particular situation.
I agree with Milevsky and Macqueen on this topic. There are situations in which an annuity will improve retirement sustainability and situations in which they will not.
I think an important question to ask an adviser, in addition to "how are you compensated?", is why they are recommending an annuity or why they are not.
Financial planner and researcher, Larry Frank, recommends you use NAPFA's Comprehensive Financial Advisor Diagnostic (download PDF) for other questions you should probably ask.
I should also note that most economists and academics strongly encourage annuitization of part of your nest egg, while financial advisors seem less inclined to do so. To quote Milevsky, "The financial economics approach is concerned with lifetime resources, constraints, risk aversion and impatience – while the financial planning approach focuses on goals, dreams and desires." Both paths can lead to an annuity where appropriate.
I'm so glad that you did caution readers to be even more wary of the pseudo-academic papers, which can seem very approachable for many of us. When I was more athletically inclined I had a friend who was constantly telling me about 'scientific' findings published in a widely circulated fitness magazine. They were patently ridiculous, intended to be attention-grabbing. I'd ask 'what was the sample size', 'what was the statistical strength of the study', or 'what was the composition of the control group'? She never did understand why I was so skeptical, because the articles 'made sense' to her.
ReplyDeleteRetirement finance seems to have more than its share of papers that present as academic research, though they are neither written by academics nor peer-reviewed.
DeleteOne can find academic papers with poor research and non-academic papers with excellent information, but I think its important to understand the authority of the source in deciding to what extent you should change your existing beliefs.
Yours is an excellent example of how this happens in other fields.
Thanks for sharing, Abby!
I use a lot of academic papers in my regular work where I need to apply their findings to the real world. Some thoughts on how to do this:
ReplyDelete1. Understand that most academics are in a publish or perish environment. Five papers arising from a single research project are almost always better than one. I usually try to use the latest and work backwards from there as necessary to see the actual data sets.
2. Many academic papers arise from Ph.D. and Masters theses. There is generally an attempt to get at least one paper out of these, regardless of whether or not the actual thesis was worth doing in the first place. So many are simply useful as somebody gathering some more data in a field. It is up to you to figure out whether or on this data set has any application to your particular problem.
3. Periodically there is a paper that comes out with a really new concept. You need to go through that in some detail to see if it is specific to a limited data set or has the potential to become more generalized. A lot of problems arise from interesting specific findings and theories from a limited data set that then become reified into a Universal Truth (Bengen's initial 4% from his first paper - his later papers had more caveats to it).
4. Every now and then, a person who has been working in a field for a long time (usually decades) sits down and puts together a "State-of-the-Art" paper. These are always worth grabbing because they are often one of the best sources to realize understand how the jig-saw pieces fit together and which ones had legs.
5. Textbooks are of highly varied quality. Most are out of date with the latest findings in some areas before they are even published. Some authors really put their heart and soul into them and others simply churn them out without a lot of thought. They are usually good for understanding the basic principles of the theory behind the problems you need to solve, but are usually too general or out-of-date to be useful on many specific problems.
These are excellent ideas.
DeleteLet me add another. Authors in retirement finance will often post about the papers on their blogs, explaining the findings in more accessible terms. They often permit you to ask questions and to see questions asked by others and the author's response. Michael Kitces and other bloggers frequently post about the research of others and his explanations are frequently more accessible than the papers themselves, though they can still be quite dense.
One of my goals for this blog is to explain and discuss research in simpler terms.
(Regarding the varying quality of textbooks, one need only consider Texas history books.)
I suppose the moral here is that you should question research findings, academic or not, rather than accept them as new universal truths simply because they were published.
Thanks for sharing!
Agree totally
DeleteWhen academics teach or publish things, lay audiences often do not engage their critical thinking skills to question what they are reading or hearing. Example: When I used to teach college courses, I would sometimes purposely say something totally preposterous to the class. More often than not, what I had said would end up in their class notes verbatim, without a challenge. And when I would ask them to take a hard, questioning look at what they just written down, they would be flabbergasted to see what they had just assumed to be factual. I'm positive that they would be a lot quicker to challenge what friends or family have to say...
ReplyDeleteLove it!
DeleteAnd that is the key point.
Many of my buddies at the Retirement Cafe are UNC profs. One told me that he has a rule he states at every first class. "If you think I'm telling a joke, you have to raise your hand."
He had learned that many students would write down anything he said.
Thanks!
This comment has been removed by the author.
ReplyDeleteThere are problems with both sides of the camp. It is not just the public; the academics and professionals are equally at fault.
ReplyDelete(1) Missing The Point Completely: Sure the general public may sometimes miss the point, but they are by no means the audience that 'matters', It is the audience of other academics and professionals with a public voice that also completely miss the points that do the most damage - by broadcasting their wrong conclusions to wider audiences.
E.g. take the research on Asset Location by William Reichenstein et al. While everyone accepts that the point of AL is to save tax dollars, his research never even attempts to measure relative outcomes, or tax dollars saved. His new idea attempts to improve the traditional mean-variance-optimization of the Asset Allocation decision. Yet it is hailed as AL research and its wrong AL conclusions are glowingly propagated by Siegel and the CPAs.
(2) Quality Of Work: I find the quality of what is referred to here as 'academic' often a disgrace. Look at all the SWR research that conveniently ignores inflation by presuming investment returns adjust up and down with greater/lesser inflation (counter-factual).
(3) Improving On The Previous Answer: How much recent academic work 'finding God in annuities' was written before (if ever) the authors read the papers by Milevsky? Some authors admit to not having got around to it yet (which in internet speak mean they never will). Go through a paper's references when you know there is a contradictory voice and you often find that voice missing. If you cannot answer a critique - ignore it.
(4) Beware the Unqualified Author: All academics and finance professionals and the public suffer from the 'I won't read what disagrees with my beliefs" syndrome. They all look for excuses to make this a valid decision. A really handy one is - "the author is a nobody". The beauty of the SSRN system is that it logs people who actually open the paper. So when you control who is notified of its existence you can see if the decision-to-ignore was made.
For six years I have been telling ALL the government, professional bodies, consumer advocates, academics and industry players that what they are teaching, testing, promoting, and using as a basis for advice regarding the benefits of the main Canadian savings tool (the RRSP) is flat out wrong. Not one has responded, and the SSRN system shows none will read the common sense evidence. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2636609
(5) Respect The Work of 'Academic' (and don't find fault because we public don't appreciate their contribution to a body of knowledge). I'm sorry but I find the work of lauded authors so bad that I created a website with the purpose of debunking their much-quoted conclusions. Take the advice about the benefits of hedging currency when investing abroad generated by Dimson et al. The common sense objection to their approach is shown at #8 http://www.retailinvestor.org/hedge.html#risk
(6) So forgive me but my respect must be earned, and what is considered 'academic' work in the personal finance field is laughable. But I have an open mind and will consider just about anyone's ideas.