When Advanced Stats Fail

If you have read this blog much at all you have probably noticed that I like using advanced baseball stats in my analysis. I love to drop wRC+ when complaining about the M's offense (Like did you know Justin Smoak has a career wRC+ of 94). I also like reading analysis rich in advanced stats; I especially enjoy sites like Fangraphs. One of the big things that advanced stat-heads like to promote is the devaluation of counting stats, like hits conceded, walks given up, and strikeouts. They also like to promote the value of rate based and projection stats like HR/FB% and FIP.  Most of all they love the uber-stat, WAR. However this love of WAR can lead to incomplete and inaccurate analysis.

WAR, or Wins Above Replacement, is an all-encompassing statistic that tries to show exactly the value an individual player. Batting, base-running, and defense is all considered in the calculation of WAR. It is an awesome concept. It has become kind of a super-stat and a go to for people trying to disprove traditional stats.

WAR is also a bit of a crutch for people that don't want to look at a whole myriad of information. Instead they can just look at one number and trust the math is good. The problem is WAR often causes people to lose context of the individual accomplishments it is rolling up.

Another favorite topic of the advanced stats community is the naivety and outdated thinking used during the voting of many season end awards such as the MVP and Cy Young. Back in 2012 the baseball world was stuck in fierce debate over who should be the MVP, Miguel Cabrera or Mike Trout. The debate here was between a triple crown winner (Cabrera) and a five tool stud (Trout). WAR said Trout was better, while traditional stats said Cabrera had one of the great seasons of the ages. In the end Cabrera won the award and half the baseball world groaned while the other half cheered. This award was seen as a battle between traditional and advanced stats. It was a microcosm of the new school vs the old school debate.

Over the next two seasons stat-heads continued to preach the glory of advanced stats. They focused their preaching on WAR because it is simple conceptually and scratches an age old baseball itch. It provides one number to rule them all. During this time advanced stats disciples like Dave Cameron and Nate Silver got big time exposure in all kinds of media and used WAR frequently in their analysis. Fans minds began to change and they got more comfortable with the math heavy world of advanced stats. But for many they embraced the change without truly understanding how to use the stats.

All this brings us to the present, where Corey Kluber just won the AL Cy Young award by narrowly edging out Felix Hernandez. The Cy Young award is award to the pitcher in each league that had the best season. Kluber, an Indians starter, had a great season posting a 2.44 ERA racking up 269 strikeouts which, according to Fangraphs, all added up to 7.3 WAR. This is an awesome season to be sure, the thing is by most other traditional measures Felix Hernandez had a better season. Felix had a 2.14 ERA, 248 strikeouts and a 6.2 WAR. He should have won the Cy Young.

For the 2014 season, WAR  is a very misleading stat. According to the Fangraphs calculation of WAR Corey Kluber had a more valuable season then both Felix Hernandez and Clayton Kershaw. Yes, you read that right. Clayton Kershaw had a lower WAR than Kluber (7.3 to 7.2). Kershaw made 27 starts had a 1.77 ERA and gave up a 6.3 H/9, and struck out 239 batter, while waking only 31. The Dodger star had the best pitching season since Pedro Martinez's 2000 campaign. Yet, despite this WAR said Kluber was better.

So, why did WAR fail to show Kershaw and Hernandez as the best pitchers in the league? Partially because it shouldn't be used as a one stop shop, but also because it relies on projected value not actual outcomes. WAR projects what should have happened rather than considering what really did happen. Let me give you an example. Two pitchers each pitch one inning and give up one homerun. With their other batters, the first gets three fielded outs, while the second gives up a hit and records three strikeouts. In this case the pitcher with three strikeouts has a higher WAR because strikeouts are more valuable and are under a pitcher control.

In addition to the counting stats, WAR also includes adjustment for the defense playing behind a pitcher and the stadium they are paying in. Recording an out with a bad fielding team behind you is more difficult than doing it with a bunch of gold-glovers. So, WAR rewards pitchers who do get an out with a bunch of Russ Davis' trying to make plays more than those with Omar Vizquels behind them. In 2014, Felix benefitted from a strong Mariners defense, while Kluber had a less than stellar crew of defenders behind him. In this case WAR gives Kluber a boost over Felix. Even though Felix gave up the least runs in the league, he doesn't get full credit for that because his defenders were too good.

As mentioned before WAR also tries to capture park effects on players outcomes. It is commonly accepted that where a player plays impacts his statistical numbers. Playing in a bandbox like the Red Sox's Fenway Park will boost batting numbers and negatively impact pitchers because long flyballs turn into homeruns. The Mariners home field of Safeco Field is considered a pitchers park. As such Safeco is detrimental to Felix's case because the outs he records are considered simpler to achieve than those at other parks.

WAR undervalues the idea that the primary goal of every pitcher is to avoid letting runs score and the secondary goal is to avoid base runners. Instead it focuses on the projected outcomes that math says should have happened. So  WAR does not provide the context for award voting, which should weigh actual results (read counting stats) heavier then projection based stats. In 2014, Felix Hernandez was the best pitcher at preventing runs and base runners. He gave up less runs, hits and walks than Kluber. What Felix did worse at were two of the three true outcomes, homeruns and strikeouts. He also had a strong defense and an offense deflating home park. As such WAR valued Kluber more highly, but Hernandez had the better season.

Basically what this all boils down to highlight that WAR should not be used as a standalone stat when voting on post season awards. The success of a player over a season needs to be considered in context. In deciding what to pay a player for potential future performance WAR is great. For determining the individual impact of a single player WAR also provides great insight. However, when determining the best player of a season WAR misses the point that counting stats capture.

The 2014 AL Cy Young award epitomized where WAR can overvalue a player compared to his actual results. In 2014 Corey Kluber had an amazing season and a crappy defense and home field behind him which inflated his WAR and resulted in a Cy Young. While King Felix also had an amazing season where he did the best in the American League at achieving a pitchers two primary goals, but was punished by WAR because he had a good defense behind him and played at Safeco field. The advanced stats, specifically WAR, appear to have swayed baseball writers voting for the AL Cy Young. Advanced stats appear to have burned a guy that deserved off season recognition.