Assessing the 2018 Indians bullpen using 3WAR and WPA

A couple of weeks ago we glanced at the performance of the Indians’ 2018 starting rotation through the lens of 3WAR — a simple average of the Fangraphs, Baseball Reference, and Baseball Prospectus wins above replacement calculations. It reaffirmed the fact that the Tribe own four All-Star caliber pitchers and underlined high hopes for Shane Bieber, the youngest of the bunch.

Today, I’d like to take the same approach to the Indians’ bullpen while also including win percentage added (WPA) and leverage statistics. It doesn’t make much sense to me to evaluate reliever performance without considering the average leverage of their “game events” and also the types of situations into which they were brought. As a reference, Fangraphs offers the following rubric for assessing a player by leverage index (LI).

Low: 0-0.85, Medium: 0.85-2.0, High: 2.0+

To give an example of relievers and WPA, Blake Treinen led the league among relievers with a WPA of 6.22 in 2018; Jeremy Jeffress and Edwin Diaz contributed 4.86 and 4.57 respectively. We also want to look at WPA/LI, which, in short, tries to show how well a player performed in regards to WPA regardless of leverage.

Combining all of these allows us to draw conclusions like, “Blake Treinen was the best reliever in 2018 and had the highest average leverage of any qualified reliever last season,” or “Robbie Erlin pitched in exceptionally low leverage situations as a reliever, but did very well when he did pitch in relief.”

Final reminder that WPA assigns credit for actual outcomes rather than expected outcomes. WPA/LI also sets 0 as league average rather than replacement level.

The chart

Not included in this chart are the relief appearances of Josh Tomlin or any of the other Indians “starters”. Between Plutko and Tomlin this is a net positive on the final numbers.

Reactions

I forgot that Brandon Guyer tossed an inning in relief. At least he didn’t contribute much to around 250 innings (when you add Tomlin and Plutko’s relief appearances) pitched at or below replacement level by the Indians bullpen.

The other thing that jumps out at me is how many relievers shared the workload this season as compared to years past. In 2017, Bryan Shaw, Cody Allen, Andrew Miller, Zach McAllister, Nick Goody, and Dan Otero all pitched more than 50 innings. Only Allen and Otero managed it this season. This is despite the bullpen throwing roughly the same number of innings in both seasons. Part of that is due to trades, part of it is due to injuries, but it does add validity to the feeling that the Indians weren’t able to lean on a reliable handful of relievers in 2018.

Next, it’s worth talking about Baseball Prospectus’s opinion of relievers. I’m inclined to agree that it is probably a little more difficult to find a replacement level bullpen arm than Fangraphs and Baseball Prospectus think; I also think you have to be drunk to the Nick Goody wasn’t at or below replacement level this season. BP’s value baseline bumps the numbers up across the board, but it doesn’t save us from the major overarching conclusion:

This is not what you want from a playoff team

The WPA results are honestly not as bad as I’d expected. The Indians rank near the middle of the pack in WPA and WPA/LI, which I attribute to the performance of guys like Oliver Perez and the addition of Brad Hand. I thought they’d be near the bottom. Among playoff teams they were still the worst; the top ten teams in WPA/LI are all of the playoff qualifiers with the exception of the Rays and the Padres. The only playoff teams outside the top ten were the Indians and the Braves.

When it comes to overall value things just get worse. By 3WAR standards they were one of the five worst bullpens in the league, which is somewhat remarkable given that they pitched the fewest innings in all of baseball.

Maybe the most troubling thing is that only five relievers managed to make any meaningful contribution to the bullpen according to 3WAR, and two of those players — Hand and Olson — pitched fewer than 30 innings for the Indians. Olson had the second-worst WPA on the team after Adam Cimber but his WAR total are buoyed by much better peripherals. Again, WPA doesn’t care expected outcomes.

From a usage standpoint I think it’s difficult to argue with Terry Francona’s decisions. The relievers he inserted into the game in the highest leverage situations overall were his most valuable relievers; the only exception to this is Oliver Perez, who probably deserved to be used more often overall and in particular with the game on the line. He pitched only 5.2 innings during high leverage situations during the season. His deployment primarily as a match-up lefty is partially to blame for this, as he made 36 appearances that lasted less than one full inning. His overall usage being relatively low isn’t just because he joined the team at the start of June; in the second half six other relievers pitched more innings.

What comes next for the Indians’ bullpen

I have absolutely no idea. Brad Hand is the only reliever guaranteed to return who was significantly better than replacement level according to 3WAR. Or WPA/LI, to be honest. The situation is hardly brighter even if you think Padres Adam Cimber is the real Adam Cimber.

3WAR isn’t kind in its assessment of the 2018 Indians’ bullpen, and neither are the leverage-based statistics. There is plenty of time left in the offseason for the Indians to improve the bullpen, but there is a lot of work to do.