“Sabermetrics, SaaS and Damn Lies” or “What the Oakland A’s Big Trade Can Teach You About SaaS Analytics”
Part 1 – Baseball, SaaS and the Rise of the Data Nerds
Baseball really boils down to one stat, score more runs than your opponent and you win, score less and you lose. Do this 90 or more out of 161 games and you will probably make the playoffs. Win 11 or 12 post season games (depending on if you need to win the one game Wildcard playoff) , winning a 5 and two best of seven game series, and you are the champions. Easy!
For around 100 years, the management of a baseball team was built around this simple fact. Statistics, while numerous, really boiled down to 4 that were focused on. Batting average and Runs Batted In (RBIs) , for offense; and wins and Earned Run Average (ERA) for pitching. These seemed like the most applicable proxies to understand when evaluating how much contribution an individual hitter or pitcher contributed to scoring or keeping the other team from scoring. Of course, the primary way that teams evaluated talent was qualitative, through scouting, ie individual observation and assessment of a player’s skill and prediction of their future success.
The Rise of Sabermetrics and Money Ball – According to Wikipedia, Sabermetics is “the term for the empirical analysis of baseball, especially baseball statistics that measure in-game activity.” Sabermetrics eschews many traditional stats as flawed, and has build an entire language and measures of its own to understand and predict the value of a player, WAR, VORP, OPS and DIPS being some of the less obscure obscure of the new stats.
While starting around the 1960, Sabermetrics gained popular notoriety with the 2003 publication of “Moneyball”, where author Michael Lewis paints a compelling picture of how Oakland A’s General Manage Billy Beane used quantitative measures to “beat” the other General Managers in baseball by being able to understand and project the value of a player to his team dramatically better, and build winning teams at a fraction of the cost of other teams, a double win, on the field and in the bottom line.
To me, it is hard to overestimate the impact of Moneyball on American business. I believe there’s a PhD thesis that could be written on how Moneyball and Sabermetrics paved the way for and paralleled the rise of the analytics driven business. When I was in business school in the early 90s, “quant jocks” were confined to the finance and operations groups, while most business were run by the sales and marketing “poets”. But the shift has finished, after all, if the data Nerds can rule the sports world, why shouldn’t they rule the business world too.
Nowhere is this more true than in the world of Software as a Service, or SaaS, where everything can be instrumented and measured. Business, like baseball is simple. Collect more money (revenue) than you spend(Cost) , and you win. However, as we’ve become better at understanding the underlying economics of SaaS subscription businesses, we’ve become SaaS Sabermatricians, and developed our own set of statistics that we use to evaluate and predict performance. We’ve developed terms like CAC, CLV, Magic Numbers, MRR, ARR and MVP, supplanting the old language of simple Profit and Loss, Revenue and Margin. In the whole, this is a great thing, we are just plain better and managing our business.
Part 2: The Trade
Consider these facts :
- On July 30, 2014, the Oakland A’s, still run by Billy Beane, had a record of 66 wins and 41 losses, the best in major league baseball, and they held a 2.5 game lead over their divison rival Anaheim Angels, who by coincidence, held the second best record in the majors at 63 wins and 43 losses.
- On July 31, in a blockbuster deal, the A’s traded star outfielder Yoenis Cespedes to the Boston Red Sox in exchange for star pitcher John Lester and outfielder Johnny Gomes.
- Since the trade, the A’s have won just 19 games and LOST 29 games, and now trail the red-hot Angels by 10.5 games with just a week left in the season.
Did Billy Beane “screw the pooch”, trading away the chemistry and offensive spark plug at the heart of what was looking like a tremendous season? Lester is one of the premier pitchers in all of baseball, and I guarantee that all of the statistics said that a team with Gomes and Lester would be expected to win more than a team with Cespedes. Billy would not have made the trade otherwise.
Yet here we sit nearly 2 months later and nothing good seems to have come from it. There are two distinctly different views, well articulated by sportswriters.
In his article, “Oakland A’s: What If Yoenis Cespedes Is Magic?” of September 13th Matt King points out that,
“Since the A’s signed Yoenis Cespedes in 2012, the team was 228-131 with him in the starting lineup and 43-69 without him in the lineup, including their 15-25 record since the trade.
And while he readily lays out the statistical case against this argument, he finishes with the rhetorical flurry of:
So it’s more than just missing Cespedes’ production. It’s more than just a slumping offense. It’s not Lester. Maybe, just maybe, Yoenis Cespedes is magic.
On the other side, Alex Hall, in “Trading Yoenis Cespedes for Jon Lester has saved the Athletics’ season” makes a compelling Sabrematrics based argument of just the opposite, arguing persuasively with number that if you think things are bad now for the A’s they would be far worse had Beane not made the trade. In fact, Hall ends with this strong endorsement of Beane,
But if Oakland does miss the postseason, then please don’t blame Billy Beane and his all-in, win-now deal. Trading Yoenis Cespedes did not ruin Oakland’s season. In fact, acquiring Jon Lester may just have been the thing that saved it. And if the A’s do make it to October and make a deep run, you better believe that Lester will be one of the leading reasons why.
So who is right? King or Hall? Magic or Data? Scientist or Poet? Beane or NOT? Did the removal of Cespedes kill all the chemistry that had the A’s believing and winning? There is no denying he was an electric spark in the A’s psyche and make-up. Or did Lester’s great performances since being added save any hope the A’s had to finish well? The stats certainly are compelling.
I suspect we will never truly know, and both arguments are actually correct. We will never know because in baseball we will never get to see the result of the 2014 season without the trade. That’s life, as it collapses from future possibity to singular reality. If the A’s go on to win the wild card and the World Series, Beane will be hailed as a genius, even if the path was difficult and harder than the counter path may have been. If they don’t naysayers will say the trade was a giant blunder, despite whatever the stats the Sabermatricians trot out. Such is part of the beauty of sport and sports fandom.
Part 3 – What’s SaaS Gotta Do With It?
Every week when I am at any SaaS client’s board or senior management meeting, I see the magic vs. numbers argument play out. The numbers show X, and we can’t explain them. We dig deeper into the numbers once more, sometimes shedding tremendous light and insight into the discussion. We pick a new path and proceed on tackling the next problem.
Just as often though we get to the end of the numbers and left at a point of intuition or “magic”. The pixels have been placed, the pages and product features have been A/B tested, and we’ve combed through the data. We are faced with a choice, usually a big one. Do we do this deal? Do we change the roadmap? Do we acquire or build this piece of technology? The numbers say one thing, or they are inconclusive. Our intuition says another.
Which do we trust, numbers or intuition? Do we stick with Cespedes or do we trade for Lester? Unlike baseball, have we exhausted all of our ability to A/B test the alternatives?
And then we get to the point where we have to make a decision and use our best judgement, balancing data and statistics with intuition and feel. I guess that’s why human’s are still in the mix after all.
This reminds me of a story from early in my career at Intel as the Intel Inside Brand Manager, a quant jocks heaven before it was in fashion. At issue was a new chip that had the shell of a Pentium and the inner workings of a 486. Should we launch it as a Pentium Lite, a 486 Plus or even a 586, or was the cost to the brand promise so high, we should scrap it altogether. The chip was attractive because it would be very efficient and low cost to produce, and even at a large discount from the Pentium pricing be extremely profitable. The numbers said do it. That is, all of the numbers except the one that was hard to quantify, the cost to both the Intel and Pentium brands. Try as we may to model this, the data was inconclusive, and piled one assumption on top of another. At the end of the day, the quants voted yes, and favored the numbers which said to launch as a “586”, taking share from AMD and other “clones”. Me, being the brand manager, voted to scrap the project, saying that while I could not quantify it, the entire program put our hundred of millions if not billions of dollars of brand equity at risk. The decision? Well, to CEO Andy Grove’s credit, I don’t think you ever saw an Intel 586 did you?