“Amazon Pays $1Billion for Business Rejected on Shark Tank” – Annie’s Newsletter, March 9th, 2018

Sign Up to Receive Annie Duke’s Weekly E-Newsletter and Updates Here

Getting past “right-and-wrong thinking” to a more nuanced evaluation of decisions

Amazon is buying Ring, a smart-doorbell and home-security company, for at least $1 billion.

That Ring’s founder, Jamie Siminoff, failed to secure an investment on Shark Tank in 2013 at a $7m valuation is a staple of the stories on the acquisition. @CNBC, for example, had this typical headline: “Amazon is buying Ring, a business that was once rejected on ‘Shark Tank.’”

The implication, almost universally, is that those investors made a massive mistake in passing. Some stories did more than imply it:

  • “Had one of the Sharks accepted his offer … today that investment would be worth $120-$180 million. Whoops!” – CelebrityNetWorth.com.
  • “Mark Cuban said he couldn’t see the company being worth $80 to $90 million. Well, by last year Ring was worth $460 million. And today’s billion dollar Amazon deal amplifies the sharks’ mistake.” – Observer.com.
  • “’Shark Tank’ moguls … let [Ring] slip through their hands. [Kevin O’Leary] talked to us about whiffing on Ring ….” TMZ.com.
  • “’Shark Tank’ can be wrong.” – Dallas News.

Resulting much?

This is a good reminder of how judging a decision solely on the quality of the result ignores many more relevant factors in evaluating the initial decision:

  • This was an expensive business to develop. Siminoff said he had eight people working for a month on the Shark Tank presentation, and that was after the company was already doing $1 million in annual sales. (Siminoff thought his business was finished after the rejection but the publicity quickly tripled his company’s sales.)
  • Siminoff’s company burned through a lot more money to get where it is today. Virgin Group’s Richard Branson subsequently invested, according to at least one report, $28 million in the company. Clearly, several more layers of investors had to contribute additional money to turn the company into something Amazon.com valued at $1 billion.
  • follow-up CNBC story suggests that’s how the Shark Tank investors evaluated the opportunity, based on what they knew back in 2013. Mark Cuban: “I would pass on Doorbot again if given the chance …. I have a fundamental aversion to companies that require raising hundreds of millions of dollars to do less [than that] in revenues.” Kevin O’Leary: “I like [Siminoff] a lot. I thought he was a great sales guy, but at that time his valuation was insane.”

Whether O’Leary (who made an offer on the show but on sufficiently different terms that Siminoff rejected it) made the correct decision is impossible to tell solely from the company’s subsequent success.

But his final assessment is wise: “I can’t go looking back –  you can’t second guess yourself. You have to do what you think is right in the moment …. For that time, I made the correct decision.”

Thinking in Bets excerpt on BehavioralScientist.org

Recently @BehScientist published “Redefining Wrong in Poker, Politics, and Beyond,” an excerpt from Thinking in Bets.

Part of our relationship with uncertainty pushes us to treat predictions as certain pronouncements, even when the prediction is clearly given as a probability. Decades of betting money based on probabilities in poker taught me that a decision about the future is rarely a sure thing, so you have to make your best prediction and take your chances.

That’s relevant in all our decisions about the future.

Who’s at fault for all this right-and-wrong thinking about probabilities? (Probably everybody)

I talk a lot about the problems that come when pundits take predictions that are necessarily probabilistic and treat them as declarations of certainty. @WSJ had a different take on this, from @RolfeWinkler and @jdlahart, explaining how declarations of probability can protect the pundits from being judged as wrong, “How Do Pundits Never Get It Wrong? Call a 40% Chance.”

The “40% rule” is apparently the sweet spot for forecasting a significant economic or financial change. As market strategist Peter Tchir said, “Get it right and you can say ‘See, I was telling everyone it could happen,’ he says. ‘Get it wrong and you can weasel your way out: ‘I didn’t say it was likely, I just said it was a strong possibility.’”

I hadn’t really honed in on how expressing a prediction as probabilistic (40% or otherwise) might be used as a shield against being wrong.

This article offers an interesting new wrinkle for me.

The common thread, in my thinking about this issue and the Journal’s reporting, is that many things can short-circuit an exchange of ideas about a prediction.

One is acting as if one outcome or the other is a certainty (or, after an outcome, treating the result as an inevitability).

Another, and they quoted Phil Tetlock (@PTetlock) on this, is to assign a probability but not with the goal of accurately assessing the chances. This includes using vague terms instead of percentages. It also includes using percentages without explanation of how you came to the assessment in some way where others could evaluate the forecast or improve future forecasts based on knowledge of your process.

Let’s all recalibrate, especially on throwing Nate Silver under the bus

Several points about the Journal article advanced the idea of truthseeking through making and evaluating predictions, as well as things that undermine the exercise.

This quote about Nate Silver (@NateSilver538), however, befuddled me.

Silver didn’t repeatedly give Trump zero chance. “The Republican primary” was six months of contests all over the country, with Silver making probability estimates on a rolling basis, both for individual state contests and for the Republican nomination.

On top of that, I can’t figure out how he could be criticized simultaneously for giving Trump “no chance to win” (in the primary) and giving Trump a 29% chance of winning (in the general).

Damned if Nate Silver does, damned if Nate Silver doesn’t.

Nate Silver has a body of predictions (actually, a body of instances of aggregating and modeling data from numerous polls) and his track record is pretty darn good. Lumping him in with the 40-percenters seems unfair.

If their point is that we should do a better job evaluating predictions, or we shouldn’t let pundits get away with vague or meaningless probabilities, Silver is clearly trying to do those things.

For a deeper dive on forecasting, I highly recommend this episode of Farnam Street’s podcast with Phil Tetlock (@PTetlock).

Including a Degas painting found, literally, under a bus

I’ve collected news accounts of a number of remarkable items recently found:

(1) A stolen Edgar Degas painting, “Les Choristes” – Reported stolen from a museum in Marseille, France, it was found in a suitcase in a bus’s luggage compartment during a random search by customs agents at a rest area in Seine-et-Marne. According to the article in @SmithsonianMag, “no one on the bus was willing to claim the bag.”

(2) The birth of a supernova – @Nature recently reported the first observation of the very early stages of a supernova. Victor Buso, an amateur astronomer in Rosario, Argentina, took a series of images of a distant galaxy from his rooftop observatory with a large telescope and captured something never before observed: the very early stages of a supernova.

(3) A missing NASA weather satellite – It’s pretty likely my scattershot reading habits are responsible but, based on scattershot, February was a great month for amateur astronomers. At the beginning of the month, @WashingtonPost profiled Scott Tilley, who, using radio equipment in his home office in British Columbia, found a missing NASA weather satellite.

(4) A lock of George Washington’s hair – found by a librarian in a 1793 almanac on the third floor of the Union College library, the subject of an interesting article in @NYTimes. (Fun facts: over 100 institutions claim to have some of Washington’s hair; Topps issued three trading cards that included a strand of his hair.)


From @AkiyoshiKitaoka:

Sign Up to Receive Annie Duke’s Weekly E-Newsletter and Updates Here