Pirates in Week 6: another look at bullpen management

Coming off their second-worst week of the season in week five, the Pirates turned it around in Week 6 and posted some of their best all-around numbers. There is a lot to get to in this week's update, including a new bullpen management study, so let's jump right in.

Run Scoring (Pirates above N.L. average for first time this season)

For the fifth week in a row, the Pirates scored an above-league-average number of runs (blue line above purple line). This raised their season RS/G to 4.14 (red line). Their run scoring is above National League average for the first time this season (red line above purple line). They are currently 1.28 RS/G above their 2012 pace (gap between red line and green line).

(Click all tables to enlarge)

Run Prevention (Pirates allow their lowest RA/G of the season this week)

Coming off their worst run prevention week of the season, the Pirates posted their best run prevention numbers in Week 6.

The Pirates allowed just 2.17 runs per game this week (blue line). This lowered their season RA/G to 3.76 (red line). Their run prevention is now only .02 over 90 percent of league average (gap between red line and purple line). They are currently only .14 RA/G over their 2012 pace (gap between red line and green line).

(An explanation of why we are tracking the offense to league average and run prevention to 90 percent of league average is here. All league averages exclude the Pirates.)

Win Probability Added (Starting pitching shines)

As measured by the statistic Win Probability Added, this week the starting pitchers contributed one win above average, the bullpen .07. (Last week the Pirates' starting pitchers cost the team -1.05 wins.) The offense subtracted .07 wins. (Interpretation: If you add all the WPAs together you get 1.0 win above average. An average team would go 3-3. So, one win above average is equal to 2 wins, or 4-2 record for the week. WPA explained.)

In Week 6, the starting pitchers deserved the most credit for one win, the bullpen for one, and the offense for two wins. Most importantly, the starting pitcher contributed positive WPA scores in each game played this week. The table below shows this week's WPA game scores:

For the season the distribution of credit and blame for wins and losses breakdowns as follows:

The six positive contributions from the starting pitchers this week raised their season positive / negative contribution ratio to +3 (20 positive WPA contribution to 17 negatives). The bullpen and offense continue to be solid. The five positive contributions from the bullpen increased their positive / negative ratio to a very good 25 positive / 11 negative. The offense added four more positive contributions this week. Finally, for the first time this season, the starting pitchers have more "most credit for wins" than they do "most blame for losses."

For the season, all three facets of the team have contributed positive WPAs in a game 13 times (i.e. team wins). In five games the starting pitching, bullpen and offense have all had negative contributions (i.e. team losses).

Matching Relievers to Leverage

On Friday (May 10), I posted a study that ranked National League teams according to how "leverage-sensitive" they were in deploying their relief pitchers in 2012. (The original post is here. It includes a more detailed discussion of this topic and the 2012 results.) In what follows, I update and revise that study to provide a snapshot of leverage sensitivity a month and half into the 2013 season.

The research question

Baseball researchers have long criticized modern bullpen management. Basically, the argument is this: managers should have relief aces, not closers. Relief aces should be deployed in high-leverage (high-leverage = high-pressure) situations, whenever those situations arise. This means that aces may be used in the seventh, eighth, or ninth inning. They are not locked into ninth-inning appearances.

At the end of the season, the relief ace should have the highest average entering leverage on the staff. By "entering leverage," I mean the measure of leverage at the moment he takes the mound. Since relief aces should be most often used to extinguish high-leverage threats, they should have the highest entering leverage on the staff. Ideally, at the end of the season, the relief ace should have the highest entering leverage, the second-best reliever the second-highest entering leverage, and so on.

In light of recent criticism of how Pirates manager Clint Hurdle's manages his bullpen as it relates to leverage ("Clint Hurdle and Pirates' Bullpen: I give up"), I was interested to find out if he was significantly different from other managers in terms of matching his pitchers to leverage. Basically the idea is that the Pirates best relief pitchers should have the highest leverage scores.

What I did

1. Collect all the relief pitchers who had at least 7 innings pitched in 2013.

2. Download each pitcher's projected FIP statistic (ZiPS) and gmLI (entering leverage) from Fangraphs.com. (Important: I am using projected FIP instead of current season ERA because relievers' ERAs are so volatile. Projections give us a better idea of a pitchers underlying talent and, more importantly, they likely coincide with managers' expectations, which obviously shape their decision making.)

3. Created scatter plots and ran linear regression trendlines through them and calculated R-Squared.

Interpretation of Results

The higher the R-squared, the more effectively a team matched its relief pitchers to leverage.

Caveats

We need to be careful about drawing any firm conclusions from these results. We are dealing with very small sample sizes this early in the season. What the data provides is a snapshot of where things stand as of Friday, May 10. I plan on updating this study at the All-Star Break and after the season.

It is certainly the case that there is more to bullpen management than just responding to leverage. Matchups matter, as do considerations of overuse. In short, managing a bullpen is much more complex than just calculating leverage and throwing the appropriate pitcher out there. I don't want to be interpreted as suggesting managers should only look at leverage.

Results

First, a scatter plot of all relief pitchers in the MLB with seven or more innings pitched.

As you can see, the trendline is moving in the expected direction. As project FIP increases, average entering leverage decreases. The R-Squared is not terribly strong, but it's not nothing either, .28.

Below is a table of the R-squares for every National League and American League team so far this year. The Pirates rank second-lowest in the National League (third-lowest in the MLB). The Chicago Cubs and Kansas City Royals lead their respective leagues.

Here is a look at the team with the highest R-Squared (.92), the Chicago Cubs.

the Detroit Tigers have the lowest R-squared (.000). (I added Jose Valverde even though he only has pitched five innings.)

The Pittsburgh Pirates (.02). Moving left to right, the dots represent Grilli, Melancon, Watson, Hughes, Mazzaro, Morris and Wilson.

Conclusion

The data suggest the Pirates are deploying their bullpen resources in a leverage-insensitive manner. There may be a couple of explanations for this. First, ZiPS is not very optimistic about Justin Wilson (5.26 projected FiP); Clint Hurdle, with some early-season justification, sees things differently. Second, ZiPS likes Vin Mazzaro (4.25 FIP) more than Tony Watson or Wilson, yet Mazzaro has largely been relegated to a mop-up, low leverage role. Third, Watson and Wilson are the two left-handed relievers in the Pirates bullpen and, as such, are being deployed to get high-leverage outs vs. opposition left-handed hitters. Finally, the Pirates may be facing a higher-than-average number of high-leverage situations in the sixth and seventh inning. Since Hurdle likes to only use Mark Melancon and Jason Grilli in their defined roles (eighth inning setup, ninth inning closer), they may be missing many high-leverage situations.

The Pirates' low leverage sensitivity score, therefore, may indicate a difference between ZiPS and Clint Hurdle's evaluation of his bullpen. Or, it may be that Clint Hurdle is more matchup-sensitive (rather than leverage-sensitive) than his peers. Whatever the case may be, this is an interesting enough topic to keep an eye on and revisit in the future.

--------------------------------

Consider following me on Twitter @DavidManel and Bucs Dugout at @BucsDugout

(Much thanks to Tom Tango and the commenters on his blog for their useful suggestions/critique of the first bullpen study.)