The following may be of very limited interest, but what the heck. On the Unofficial Pirates Message Board, user No. 9 wonders what Chris Shelton must think of the Rule 5 draft process.

It's a great question. Shelton was selected from the Pirates in the Rule 5 draft after the 2003 season. Like many Rule 5 draft picks who stick with their teams, he played sparingly the following year, and also spent some time "rehabbing" at Class AAA. He collected only 108 total at bats on the season, then returned to Class AAA in 2005 before being called up. He was one of the Tigers' best hitters in 2005 and then sparked their hot start with a spectacular April in 2006, but he completely lost it after that and got sent to the minors and then kicked off the 40-man roster.

I come neither to criticize nor defend the Pirates' decision to leave Shelton off the roster in the first place. That's been rehashed here a million times. Instead, I wonder whether the benefits of the Rule 5 are worth the cost to players like Shelton.

What the Rule 5 draft is supposed to do is to keep teams from stockpiling tons of major-league-ready talent in the minors. It allows teams to protect a certain number of players from being selected despite their still being in the minors, but subjects others to a process in which another team can claim them and keep them if they leave them on their active roster all year.

Does the process work? That is, does the process allow teams to identify guys who are ready to play in the majors - and actually allow them to play there?

Using this site, I looked at every Rule 5 draft from 1998 to 2006 for successes. I defined a success as any player who pitched at least 40 innings in the majors the next year with an ERA below 5 (and yeah, I know ERA isn't the best stat for pitchers generally and relievers especially, but it's a way to keep things simple) or had at least 200 at bats in the majors with an OPS above .700. It didn't matter to me which team the player was with at the time.

Anyone who didn't meet those criteria is described as a "flop," even if that player stuck with his new organization and eventually had success. After all, if a team needs to hide a Rule 5 player on its bench or on "rehab" assignments in the minors, then the Rule 5 draft isn't doing its job. So in describing someone like Johan Santana as a flop, I'm obviously not saying he was a bad Rule 5 choice. I'm just saying he's not an example of the Rule 5 draft doing what it's designed to do.

(By the way, in some cases I'm dealing with pretty spotty data. I mostly used Baseball Reference, which is pretty reliable, but I'm worried that I somehow missed something or counted the wrong player if a player shares a name with another minor leaguer. So let me know if I missed something.)

1998

Successes: Scott Sauerbeck

Flops: 12

Notes: The Cardinals were probably pretty happy with catcher Alberto Castillo, whose OPS was a bit too low to make the cut.

1999

Successes: None

Flops: 17

Notes: Derrick Turnbow, who pitched 38 innings for Angels and posted a 4.74 ERA, narrowly missed the cut. Johan Santana was the second pick in the draft, but he posted a 6.49 ERA in 2000. Chris Reitsma was the last pick in the draft.

2000

Successes: Jay Gibbons, Jose Nunez 

Flops: 8

Notes: Endy Chavez, taken with the fourth pick, hit .208/.238/.234 in 77 at bats for the Royals. 

2001

Successes: Corey Thurman

Flops: 11

Notes: Miguel Asencio pitched 123.3 innings with a 5.11 ERA for the Royals, so you can chalk that up as a success if you want to. Jorge Sosa pitched 99.3 innings with a 5.53 ERA for the Devil Rays. 

2002

Successes: D.J. Carrasco, Aquilino Lopez, Javier Lopez, Luis Ayala, Michael Neu, Chris Spurling, Matt Ford

Flops: 21

Notes: Wil Ledezma posted a 5.79 ERA in 84 innings for the Tigers. Matt Roney had a 5.45 ERA in 100.7 innings, also for the Tigers. Shane Victorino, Ronny Paulino and Adrian Brown were all chosen in the late rounds.

2003

Successes: Jeff Bennett, Luis Gonzalez (the Rockies middle infielder, not the Arizona slugger)

Flops: 18

Notes: You could probably also add Hector Luna to the "success" category; he posted a .249/.304/.364 line in 173 at bats for the Cardinals.

2004

Successes: Andy Sisco

Flops: 11 

Notes: Marco Carvajal pitched 53 innings with a 5.11 ERA for the Rockies. D.J. Houlton pitched 129 innings with a 5.16 ERA for the Dodgers.

2005

Successes: Dan Uggla

Flops: 11

Notes: Fabio Castro pitched 31.7 innings with a 2.27 ERA for the Rangers and Phillies. Victor Santos pitched 115.3 innings with a 5.70 ERA for the Pirates. 

2006

Successes: Joakim Soria, Josh Hamilton, Jared Burton, Kevin Cameron

Flops: 15

Notes: Jesus Flores hit .244/.310/.361 in 180 at bats as the Nationals' backup catcher. Jason Smith and Josh Phelps were both useful in the majors, but in more limited roles.

CONCLUSIONS:

Teams have had more success in the past two years identifying players like Uggla, Soria, and Hamilton who are not only have immediate success in the big leagues but are likely to stick there and continue to have success once their Rule 5 year is over. Whether that's the result of improved player evaluation by some franchises or just a sample-size fluke is open to debate.

What's not open to debate is that the Rule 5 draft does not work for the large majority of players selected. That is to say, it does not identify players who are ready to succeed in the majors and find playing time for those players.

In many cases, that simply means the player is offered back to his old team during Spring Training, or after a few poor innings. One or two more bad outings, and Evan Meek is likely to fall into that category. So in those situations, it's relatively harmless, at least to the player.

In other cases, though, the Rule 5 draft isn't harmless at all. How many Chris Sheltons have spent a year - or even a couple months - sitting on the major league bench when they could be playing in the minors? In Shelton's case, 2004 probably would have been a critical year for him. He'd hit fantastically in Class  A ball and was promoted to Class AA down the stretch in 2003. He had yet to master that level, and the beginning of the 2004 season would've been a critical time in his development as he tried to figure out AA and, hopefully, AAA. Instead he spent the year not playing baseball much at all. Jose Bautista, too, missed an entire year of development.

It's impossible to guess what would have happened if things were different. It's nearly certain, though, that many Rule 5 players would have had better careers if they hadn't been drafted. How many Joakim Sorias or Dan Ugglas do there have to be to offset the damage done to those other guys' careers?

Even the cases of many of the "successes" are striking, I think. In 2002, there were seven "successes." Of those, only Luis Ayala has had a relatively good, sustained career. (Javier Lopez may yet; the others probably never will.) Maybe they just weren't very good to begin with, which isn't a data point in favor of the Rule 5. Or maybe the process messed them up in some way by assigning them to roles they weren't really ready for. Matt Ford, for example, was a starter in the minors until Milwaukee used him as a reliever in his Rule 5 year, then tried to let him start again in the minors once his Rule 5 year was over. The Blue Jays did the same to 2001 Rule 5 "success" Corey Thurman.

From looking over these drafts, it appears that the Rule 5 draft isn't fulfilling its purpose. This is not to say that the Pirates, or any other team, should avoid choosing in the Rule 5 draft as long as it exists. It's probably to the Pirates' advantage to pick an Evan Meek and see what happens. However, it is probably not to Evan Meek's advantage to see what happens - he'd be better off honing his craft in the minors. And as we saw yesterday, the Rule 5 draft often fails to help teams put the best baseball players on the field. Sometimes it does, but more often than not, what happens is that an unripe Jose Bautista sits on the bench for a year and limits his managers' tactical options. In a worst-case scenario, you have someone like Meek in an important situation out there trying to do something he can't yet do.

Maybe there need to be changes to the Rule 5 draft. I'm not quite sure what those should be, but just as a starting point for discussion, maybe it would be a good idea to: 

1. Turn the 40-man roster into a 38- or 37-man roster. This would limit the number of roster spots teams could use on players they don't plan to use in the majors, and thus increase the number of Rule 5-eligible players who are ready for big-league roles.

2. Raise the price necessary to select a Rule 5 pick to, say, $250,000. This way, teams would have to be more certain of a Rule 5 pick's ability to play in the majors. (It might, however, have the unfortunate side effect of some terrible franchise like the Marlins leaving players unprotected on purpose just to collect free money.)

What does everyone think?