Cleveland Indians spring training statistics - do they matter?

"Spring training numbers just don’t mean a thing. At all. Anything…Ignore the numbers coming from the Cactus and Grapefruit Leagues." Dave Cameron (Fangraphs) in February 2010.

Cameron is  far from the only baseball journalist to claim that spring training stats don't matter one jot, but Dan Rosenheck believes that this anything but the case.  In an article in The Economist last month Rosenheck makes a persuasive case that some ST stats do matter. (There is also a short interview with Rosenheck here, in which he quickly runs through his model, and he also discusses it from around 14:00 to 42:00 in this SB Nation White Sox podcast).

While recognizing that ST games clearly aren't the same as regular season, since players show up in a variety of fitness conditions, often face minor league opposition, while testing out an improved swing or new pitch wrinkle etc., Rosenheck states that "the claim that spring training numbers are useless is wrong. Not a little bit wrong, not debatably wrong – demonstrably and conclusively wrong."

So which spring training stats do actually matter?

Some baseball statistics become reliable much quicker than others, requiring remarkably small sample sizes. In his recent presentation at the MIT Sloan Sports Analytics Conference (for which you can find the slides here)  Rosenheck demonstrates that the following ST stats show good correlation with regular season stats (shown by R², where an R² of 1.0 would be perfect correlation)

Hitters (Min 50 ST PA/500 Regular Season PA)

Walk rate: BB-IBB/(PA-SH-HBP-IBB)

ST vs. Regular Season: R² of 0.23 (p.11 of Rosenheck's slides)

Strikeout rate: SO/(PA-SH-HBP-BB)

ST vs. Regular Season: R² of 0.44 (p.12)

Isolated power on contact (aka ISOCON): 2B+(2*3B)+(3*HR)/(PA-SH-HBP-BB-SO)

ST vs. Regular Season: R² of 0.28 (p.13)

Steal attempt rate: SB+CS/SBO

ST vs. Regular Season: R² of 0.50 (p.13)

Starting Pitchers

Walk rate: BB-IBB/(BF-HBP-IBB)

ST vs. Regular Season: R² of 0.18 (p.14)

Strikeout rate: SO/(BF-HBP-BB)

ST vs. Regular Season: R² of 0.27 (p.15)

Ground ball rate:GB/(BF-HBP-BB-SO)

ST vs. Regular Season R² of 0.32 (p.15)

But do they really matter?

As we have just seen, spring training results do appear to have some predictive value (notwithstanding the fact that correlation doesn't necessarily imply causation), but are they valuable? Do we already possess the information that ST stats provide from other sources, thus making them essentially redundant? To determine whether ST stats add anything to what we already know, Rosenheck decided to compare them to a common projection system (Dan Szymborski's ZiPS). If adding a ST stats component to ZiPS leads to better forecasts than using ZiPS alone, then we can reasonably conclude that ST stats do matter.

Bear in mind that "the less we know about a player, the more weight we want to place on new information like spring training stats" (Rosenheck). For obvious reasons, ZiPS is much more effective for predicting veteran performance than it is for rookie performance. Thus Rosenheck's projection method gives much more weight to ST stats for rookies than for veterans.

Overall, Rosenheck calculates that for offense, adding ST stats to ZiPS forecasts increases their accuracy by a not insignificant 4.6%, while also demonstrating a much greater effect for rookies and players coming off long injury breaks. "The biggest impact is on stolen bases, probably because stolen base attempts reflect the manager's intentions as well as the player's performance" (Rosenheck).

However, the effect of incorporating ST stats on forecast accuracy for starting pitchers is actually twice as large as that for offensive players: "the difference for a starter between a great spring and a terrible one is 75 points of forecast ERA." 

So what does all this mean for the Indians?

Spring training stats that matter for Tribe hitters

Only eight Tribe hitters mustered the "required" 50 PAs in ST. Let's look at each of them in turn, listed in descending order of plate appearances. The ZiPS projections can be found here.

"QUALIFIED" HITTERS (50+ PA)

	Walk %	SO %	ISOCON %	Steal %
SANTANA (76 PA)
ZiPS	14.89	21.92	24.63	2.36
ST	10.67	23.88	21.57	0.00
ST vs. ZiPS	NEGATIVE	NEUTRAL	NEUTRAL	NEGATIVE
RAMIREZ (67 PA)
ZiPS	5.61	12.45	11.04	19.14
ST	3.08	15.87	9.43	17.14
ST vs. ZIPS	NEGATIVE	NEGATIVE	NEUTRAL	NEGATIVE
BOURN (61 PA)
ZiPS	7.02	25.23	13.55	14.59
ST	6.56	19.30	13.04	3.48
ST vs.ZiPS	NEUTRAL	POSITIVE	NEUTRAL	NEGATIVE
CHISENHALL (61 PA)
ZiPS	5.91	20.85	19.83	2.25
ST	8.33	18.18	22.22	0.00
ST vs. ZiPS	POSITIVE	POSITIVE	POSITIVE	NEGATIVE
MURPHY (56 PA)
ZiPS	7.38	15.22	15.38	4.85
ST	5.36	3.77	1.96	0.00
ST vs. ZiPS	NEGATIVE	POSITIVE	NEGATIVE	NEGATIVE
GOMES (53 PA)
ZiPS	5.17	25.45	24.70	0.63
ST	12.00	31.82	30.00	0.00
ST vs. ZiPS	POSITIVE	NEGATIVE	POSITIVE	NEUTRAL
AVILES (51 PA)
ZiPS	3.70	14.10	13.13	13.08
ST	2.00	2.04	0.00	13.33
ST vs. ZiPS	NEGATIVE	POSITIVE	NEGATIVE	NEUTRAL
SANDS (50 PA)
ZiPS	8.95	34.55	15.45	1.62
ST	8.00	23.91	20.00	0.00
ST vs.ZiPS	NEUTRAL	POSITIVE	POSITIVE	NEUTRAL

Overall, there are slightly more negatives than positives relative to ZiPS among these players. However, players such as Santana and Bourn can probably be safely discounted, as they are established MLB players for whom ST stats have very little significance. Of more concern (given that he is still a relative newcomer) is Ramirez's performance for these stats and the drastic loss in power for the aging Murphy. On the positive side, both Chisenhall and Sands hit much better than ZiPS would have expected in ST.

The following players don't quite meet the 50 PA threshold, but with at least 37+ PA (for the 25-man) or 45+ PA (for non-25 man) they are close enough, with just an asterisk for smaller than ideal sample size:

"NON-QUALIFIED" HITTERS (FEWER THAN 50 PAs)

	Walk %	Strikeout %	ISOCON %	Steal %
MOSS (48 PA)
ZiPS	9.38	34.05	35.95	2.32
ST	4.17	34.78	66.67	0.00
ST vs. ZIPS	NEGATIVE	NEUTRAL	POSITIVE	NEGATIVE
AGUILAR (47 PA)
ZiPS	7.30	29.53	19.27	0.00
ST	6.38	13.64	15.79	0.00
ST vs. ZiPS	NEUTRAL	POSITIVE	POSITIVE	NEUTRAL
BRANTLEY (47 PA)
ZiPS	6.88	10.91	15.44	8.26
ST	4.35	11.36	10.26	4.44
ST vs. ZiPS	NEGATIVE	NEUTRAL	NEGATIVE	NEGATIVE
HOLT (47 PA)
ZiPS	8.35	25.54	6.65	14.27
ST	10.64	19.05	8.82	21.33
ST vs. ZiPS	POSITIVE	POSITIVE	POSITIVE	POSITIVE
MARTINEZ (47 PA)
ZiPS	4.69	19.67	2.45	6.91
ST	2.13	13.04	15.00	33.33
ST vs. ZiPS	NEGATIVE	POSITIVE	POSITIVE	POSITIVE
RABURN (46 PA)
ZiPS	6.44	28.74	21.02	0.00
ST	15.22	28.21	7.14	0.00
ST vs. ZiPS	POSITIVE	NEUTRAL	NEGATIVE	NEUTRAL
ROHLINGER (45 PA)
ZiPS	6.25	21.74	4.44	1.62
ST	13.33	5.13	32.43	0.00
ST vs. ZiPS	POSITIVE	POSITIVE	POSITIVE	NEUTRAL
KIPNIS (43 PA)
ZiPS	9.40	21.82	16.70	12.70
ST	16.67	22.86	18.52	5.71
ST vs. ZiPS	POSITIVE	NEUTRAL	POSITIVE	NEGATIVE
PEREZ (37 PA)
ZiPS	9.91	33.33	15.98	1.86
ST	19.44	27.59	23.81	0.00
ST vs. ZiPS	POSITIVE	POSITIVE	POSITIVE	NEUTRAL

There are far more positives than negatives in this group. Notably, the power numbers for Moss and Kipnis are encouraging, as they have both been rehabbing injuries. The stats for Holt, Aguilar and Perez are probably more significant than any of the others, given that they have so little MLB experience, and all are positive (despite the fact that MLB has brazenly robbed Holt of 9th inning double on Mar 19, giving it to Erik Gonzalez - check 2:48:03 on the video of that game if you don't believe me!). Unless Raburn can start to show some power now that the regular season has started, he may be gone by the end of May - two extra-base hits (both doubles) in 46 PAs just doesn't cut it for a platoon "power" hitter.

Spring training stats that matter for Tribe pitchers

And what about the starting pitchers? The three categories of interest here are K rate, BB rate and ground ball rate. Unfortunately, I don't have a source for GB spring training statistics, as those figures aren't available on either MLB.com or on Baseball Reference, so here we shall focus just on K rate and BB rate, for which we will use K/9 and BB/9 as proxies (since ZiPS doesn't provide the data on the number of batters faced that we would require to calculate ZiPS forecasts for K rate and BB rate using Rosenheck's method).

STARTING PITCHERS

	K/9	BB/9
BAUER (27.2 IP)
ZiPS	8.75	3.86
ST	8.46	0.04
ST vs. ZiPS	NEUTRAL	POSITIVE
McALLISTER (25.1 IP)
ZiPS	6.89	2.60
ST	9.95	1.42
ST vs. ZiPS	POSITIVE	POSITIVE
HOUSE (23.2 IP)
ZiPS	6.18	3.13
ST	7.23	1.90
ST vs. ZiPS	POSITIVE	POSITIVE
KLUBER (23.1 IP)
ZiPS	9.86	1.92
ST	9.64	1.93
ST vs. ZiPS	NEUTRAL	NEUTRAL
SALAZAR (11.0 P)
ZiPS	10.21	3.00
ST	12.27	4.09
ST vs. ZiPS	POSITIVE	NEGATIVE
CARRASCO (9.0 IP)
ZiPS	8.39	2.57
ST	6.00	4.00
ST vs. ZiPS	NEGATIVE	NEGATIVE

Bauer's ST numbers are very promising relative to ZiPS due to the massive reduction in walk rate. Even if he does continue to give up homers (which certainly isn't a given once he gets away from Arizona), at least the damage should be limited if he can keep down the free passes.

McAllister has absolutely crushed his ZiPS regular season forecast in ST, with a terrific strikeout rate and better than expected walk rate. He's certainly had enough MLB starts (65) that you would expect ZiPS to have a pretty good handle on him by now, but McAllister is younger than Carrasco and younger than Kluber was this time last year, so a breakout season at the age of 27 (as he attempts to "do a Carrasco", as Jeff Sullivan wrote about here) isn't entirely infeasible.

On the face of it, House's 5.70 ERA looks bad, but his ST BB/9 and K/9 rate peripherals have both been notably better than ZiPS projects for the regular season, which augurs well. Meanwhile, Kluber's ST K/9 and BB/9 are both very close to the ZiPS forecast.

However, although Salazar struck out a bunch of guys in ST, his control was poor, as evidenced by a BB/9 of over 4. If Salazar has been poor in terms of peripherals then Carrasco has basically been terrible. Let's hope that this is just a temporary blip caused by small sample size as a result of his recent illness and subsequent paternity leave.

Here we are of course discussing projected performance relative to ZiPS. However, ZiPS is far from bullish about the Tribe's rotation overall, forecasting Kluber at 4.8 WAR and everyone else below average: Salazar at 1.9, Carrasco at 1.4, Bauer and McAllister at 1.0 and House below replacement at -0.1 WAR. In my view, it would be most disappointing if the Tribe doesn't manage to out-perform the combined ZiPS WAR numbers for this group, which would almost certainly mean that they would need to beat the K/9 and BB/9 ZiPS numbers overall.

Conclusion

The good ST performances of some of the Tribe's less well established players (Holt, Aguilar, Perez, McAllister and House) is probably much more significant than the slightly disappointing numbers put up by some of the veterans. If guys like Murphy and Raburn don't turn things around in the next couple of months then at least the Tribe appears to have some potential replacements on hand.

It might be interesting to revisit these numbers after the 2015 season to see whether or not the "ST-augmented" ZiPS outperformed the "regular" ZiPS.