Monday, January 8, 2018

Minimizing Regret

I'm reading Algorithms to Live By: The Computer Science of Human Decisions by Brian Christian and Tom Griffiths[1]. Following is an excerpt.
Regret can also be highly motivating. Before he decided to start, Jeff Bezos had a secure and well-paid position at the investment company D. E. Shaw & Co. in New York. Starting an online bookstore in Seattle was going to be a big leap—something that his boss (that’s D. E. Shaw) advised him to think about carefully. Says Bezos:
"The framework I found, which made the decision incredibly easy, was what I called—which only a nerd would call—a “regret minimization framework.” So I wanted to project myself forward to age 80 and say, “Okay, now I’m looking back on my life. I want to have minimized the number of regrets I have.” I knew that when I was 80 I was not going to regret having tried this. I was not going to regret trying to participate in this thing called the Internet that I thought was going to be a really big deal. I knew that if I failed I wouldn’t regret that, but I knew the one thing I might regret is not ever having tried. I knew that that would haunt me every day, and so, when I thought about it that way it was an incredibly easy decision."
Regret minimization can be a powerful tool for making retirement planning decisions. I have always used a similar approach to my critical life decisions. I wrote about it in a post some time back, but my process works like this.

I imagine myself at some point in the future long after having made the decision and I imagine that it turned out very badly. My future self then asks, "Do I still think it was a good decision? Would I make it again?" If my future self answers no, then my present self doesn't make that decision.

Even though I assume my decision turned out badly, I recognize that good decisions can have bad outcomes. I can accept bad outcomes if I made the best decision available to me at the time. A poor decision that ends well is just dumb luck.

Imagine that you are a basketball player about to take a game-winning (or losing) shot. Your shot is a low-percentage gamble but you can also pass to a teammate who has a better shot.

If you take the shot and win, you will have a great outcome from a poor decision. Try that often and you will lose a lot.

If you pass to the open teammate and he misses, you suffer a poor outcome from a good decision. Make that kind of decision often and you'll win more than you lose.

The fact that nearly all retirement finance decisions are probabilistic means that we can make bad decisions that turn out well or good decisions that turn out badly. To complicate matters, our own retirement is a one-time event. If we could have many retirements, a 90% probability of success would mean that 90% of our retirements would be successful, but we only get one. We can and should bet on the 90% probability but if we lose the bet, 100% of our outcomes (there will only be one) will be bad. When we lose the bet, the outcome won't be bad 10% of the time or only 10% bad.

Still, the better strategy is to consistently make good decisions or "the best bets", if you prefer. While we only get one shot at claiming Social Security benefits, for example, we will make many other retirement decisions and if we choose the 90% probability bet every time we are likely to win most of them.

Recently, a reader commented that since we can't be sure that delaying Social Security benefits will have a good outcome we really can't make a blanket assessment of the strategy. We can't make a blanket statement about the outcomes, true enough, but we can make a blanket statement about the quality of the decision.

Minimizing regret is an excellent tool for deciding when to claim Social Security benefits, assuming your financial circumstances afford you the option.

Retirees who delay claiming and die early in retirement might regret that they could have received greater benefits had they not delayed, at least to the extent that people who are no longer living have regrets.

Married retirees, however, will have surviving spouses whose survivors benefits may be limited (if they are the lower earner) by the higher-earning spouse claiming early and that spouse may not regret your decision to delay even if you do regret it.

Retirees who claim early and live a very long time will likely regret their lower lifetime benefits. Widows who live on reduced survivors benefits long after their husband passes because he claimed early might regret having let him make the financial decisions.

We might regret delaying claiming if Social Security were to be abandoned entirely by the federal government early in our retirement. I wouldn't regret my decision to delay in that scenario because I assign a low probability to my cohort losing those benefits. After that outcome, I believe I would say that I would make the same decision under the same circumstances if I had it to do over.

You, however, might not agree with that assessment or might be substantially younger and have a different outlook. Regret minimization can be subjective and risk can be dependent on one's life expectancy.

Regret can be a personal thing, though it can often be measured objectively in dollars. The dollar amount of regret can be defined as the difference between the outcome you expect and the outcome that would have resulted from clairvoyance, ie., from knowing the best answer. If the best possible strategy would have resulted in a $100 profit and yours results in $90, you have $10 of regret.

Minimizing regret can help you make better retirement planning decisions.
[Tweet this]

One way to look at the Social Security claiming decision is to consider how much you or your surviving spouse would regret that decision in various scenarios and to make the choice based on avoiding scenarios with the greatest regret. This process won't favor delaying claims for every person in every scenario, but often it will.

Minimizing regret doesn't have to be the only tool you use for a specific decision but it may provide an additional perspective. Optimization tools like MaximizeMySocial Security[2], Financial Engines[3] or AARP[4], for example, also provide useful input.

Likewise, Social Security claiming isn't the only retirement decision for which regret minimization might be useful. Let's look at asset allocation.

I am thoroughly enjoying Algorithms and plan to read it again as soon as I finish. Be forewarned, however, that if you're not a computer scientist, you might be happier reading tax tables. That having been said, here's another excerpt that I enjoyed.

Harry Markowitz won the Nobel Prize in Economics for developing Modern Portfolio Theory (MPT). MPT calculates an "efficient frontier" of portfolio allocations that maximizes portfolio returns for various levels of market risk.

MPT determines an optimal asset allocation based on risk tolerance, market volatility, risk-free rates and the covariance of asset classes.

How did the father of Modern Portfolio Theory allocate the assets in his own retirement portfolio?
"I should have computed the historical covariances of the asset classes and drawn an efficient frontier. Instead, I visualized my grief if the stock market went way up and I wasn’t in it—or if it went way down and I was completely in it. My intention was to minimize my future regret. So I split my contributions fifty-fifty between bonds and equities."
Interestingly, a 50% equity allocation falls into the sweet spot of several very different research strategies. Using a complicated simulation strategy, Gordon Irlam found that the 95% confidence interval for the optimal asset allocation ranges from 10% to 80% equities.

Using a much simpler simulation strategy, William Bengen's work on sustainable withdrawal rates shows optimum asset allocations between about 35% and 60%.

In a paper entitled Nearly optimal asset allocations in retirement[5], Wade Pfau concludes, "with Monte Carlo simulations based on historical data parameters, a 4.4 percent withdrawal rate for a 30-year horizon could be supported with a 10 percent chance of failure using a 50/50 asset allocation of stocks and bonds. But the range of stock allocations supporting a withdrawal rate within 0.1 percentage points of this maximum extend from 27 to 87 percent."

That's a lot of research to find answers consistent with "My intention was to minimize my future regret. . . So I split my contributions fifty-fifty."

The Markowitz story also struck a chord with me on a topic to which I have been giving a great deal of thought lately.

We have faster computers, better algorithms, and more in-depth research into retirement financial planning but very little empirical evidence to show how much they actually improve outcomes.

There is talk of "evidence-based" strategies, but retirement research doesn't work like medical research. We can't ask one group of retirees to use a portfolio-spending strategy and a control group to buy annuities and compare the results after 30 years. Even if we could, market uncertainty means we can't expect similar outcomes the next time we run the experiment.

What we will find is evidence of uncertainty.

If Harry Markowitz thought that a fifty-fifty regret-minimizing strategy was preferable to mean-variance optimization, I won't argue.

Next, let me tell you about That Time I Maximized Regret.


[1] Algorithms to Live By: The Computer Science of Human Decisions by Brian Christian | Goodreads

[2] When Should I Take Social Security to Maximize My Benefits? | Maximize My Social Security

[3] Social Security Retirement Calculator | Retirement Readiness

[4] Social Security Calculator – AARP

[5] Nearly optimal asset allocations in retirement, Wade Pfau.


  1. Nice job on this Dirk. One additional twist on this per Daniel Kahneman is that we feel more regret for actions taken that don't work out than when we don't take action and things don't work out. For example, one feels more regret if one increases a stock allocation and the market tanks than if one leaves a stock allocation as is and the market takes off. So one way to minimize regret is not to do anything. But that's probably not a great idea. Perhaps a better idea is to do things gradually, e.g., gradually increase a stock allocation. It might be financially optimal to do it all at once, but better to spread changes over time to reduce potential regret.

  2. Good point. Thanks, Joe!

    I regret not mentioning Kahneman in the post.

  3. This excellent post was very timely for me!

    I got really nervous a few months ago and cut back on equities, taking a bit of a tax hit but being rewarded by less worry. The markets then commenced to rise even faster, and now everyone is crowing about how the tax reforms will boost them through 2018, as if there was no possible direction but up. So maybe I should rethink that shift?

    This morning I decided I prefer the risk of earning less than I might over the risk taking a huge hit from a market drop when I am almost ready to retire early. Now, after reading your post, I’m even more comfortable with my decision to stay where I am - at 50-50. :-)

  4. In my experience, Abby, the market cares very little about "what everyone is crowing" except that when "everyone" agrees on market direction, it usually means it's about to head in the opposite direction (see Irrational Exuberance.)

    In early 2000, "everyone" said the Tech Boom would last forever. Since I had already accumulated "enough", I decided that the regret of losing a secure financial future would be far greater than the lost opportunity to earn even more than I needed. I sold all my options.

    Scores of my colleagues subsequently experienced the regret of having had financial security "on paper" and then losing it all. They thought I was being stupid by selling.

    Hope your decision works out for you!

    Thanks for writing.

  5. A somewhat related blog on regret for retires on Vanguard was the most popular post of 2017. Here is the link:
    There are lots of great practical examples and lessons learnt that folks can learn from. Personally, my biggest regret is not living more within my means and starting to save a younger age losing the power of compounding. We have done OK with catch-up but it has been an uphill slog.

    1. Thanks for sharing! There are positive lessons, as well. Regret minimization kept me from losing my Tech Boom profits. Here's an embedded link to that Vanguard site so you don't have to copy and paste

      Thanks, Rob.

  6. Reposted for Steve Vernon. I included the link to his book, which wasn't in his original post:

    Hi Dirk

    Great post! I agree with all your insights and have applied regret in my retirement planning.

    If I delay Social Security and then die early, I experience "regret from heaven." I really can't imagine regret from heaven.

    On the other hand, if I claim Social Security early and live a long time, it's easy to imagine being alive, old, and poor, and then regretting that I claimed Social Security early. In fact, I have many older friends in their 70s and 80s who have this very regret, so it's very easy for me to imagine that regret.

    I'm a 64 year-old actuary who has given a lot of thought to retirement planning. I'm delaying my SS until age 70. Until then, I'm making just enough money to cover my living expenses, doing work I really enjoy. I don't regret continuing to work. Life is good!

    Thanks, Steve Vernon

    My 2012 book Money for Life contained an entire chapter on Win/Regret analysis as applied to retirement income planning. My thinking has evolved since then, but I still like the Win/Regret analysis.