Analytics, Technocrats, & the Distinction Between Talent Evaluation and Resource Allocation

Earlier today the Cleveland Browns hired Paul DePodesta away from the New York Mets to assume the newly created role of Chief Strategy Officer. As a key figure in Moneyball the book and the basis for Jonah Hill’s character in Moneyball the movie, DePodesta is closely associated with sports analytics.  Because of DePodesta’s association with analytics and the fact that his experience as an executive lies in baseball, the reactions to this hire were, unsurprisingly, quite strong.

  • Jason La Canfora of CBS Sports: “The Paul DePodesta hire could be revolutionary. We’re about to find out, truly, for better or worse, whether advanced analytics works in NFL.”
  • Louis Riddick of ESPN: “There is a place for objective/data driven analysis to co-exist with the largely subjective process of football team building. But it will NEVER replace it. Learn the game. First. The popularity of the sport has made many wish they could objectify the game because they are either unwilling or unable to learn the game.”
  • Albert Breer of NFL Network: “I have my doubts about the effectiveness of analytics in football. But maybe I’m like the old Moneyball scout grading guys on their gf’s.”

First, I would point out that individuals with an analytics acumen would likely disagree with La Canfora that a sample size of one team is enough to prove or disprove the utility of anything. Second, I would point out that Riddick falls into the same trap as the sabermetric skeptics of the post-Moneyball era in assuming that proponents of analytics intend for it to completely replace subjective evaluation. Proponents of analytics have, from the beginning, advocated for a balance between the two approaches.

More importantly, I think the reactions of league observers may be misplaced as a result of conflating the two primary decision-making processes that each front office must continually undertake. The first decision-making process resolves around determining which players are better than other players. This is the talent evaluation process. The talent evaluation process can be performed subjectively (scouting/film study), objectively (analytics) or based on a combination of the two. The second decision-making process resolves around determining which price to pay (in terms of salary cap space and/or draft picks) to acquire the contractual right to the performance of a given player. This is the resource allocation process. The resource allocation process involves executing transactions on the basis of the results of the talent evaluation process.  Ideally, each individual in a football operations department should have a clear role supporting one of these two processes, based on the skills and experience of the given individual.

Most NFL General Managers are hired due to their perceived expertise in the talent evaluation process. They are typically “football guys” who have risen through the ranks of scouting departments and have developed a reputation for being able to identify talented players more effectively than their peers. Most, however, do not appear to have any discernable claim to expertise in the resource allocation process. A career spent playing, studying and/or coaching football does not prepare oneself to efficiently allocate $150 million worth of salary cap space and numerous draft picks across dozens of roster spots while competing against 31 opponents under conditions of CBA-enforced scarcity of resources. The two processes require completely different skill sets, but the NFL has historically operated as a technocracy in which scouting technocrats typically perform both roles.

League observers seem to be under the impression that the primary motive for the Browns hiring DePodesta is to convert the organization from employing a subjective talent evaluation process to an objective talent evaluation process. In other words, to cause the organization to bring analytics and scouting onto equal footing. While this may be part of the motivation for the hire, I suspect that the primary motivation was to secure the services of an executive with a claim to expertise related to the resource allocation process. DePodesta would seem to fit this description. The thesis of Moneyball, after all, was that a team can gain a competitive advantage by developing methods to allocate its resources more efficiently than other teams.

If this is the case, DePodesta can make a positive impact on the Browns without making any changes to the talent evaluation process (although I do not doubt that he will make changes). Scouting and film study can continue to reign supreme over quantification of performance and talent. The key difference will be that after hearing the Browns’ scouts explain why think Player X is better than Player Y and how much better they think Player X is than Player Y, DePodesta will have the authority to decide if the price necessary to acquire Player X is worth the incremental cost as compared to the price necessary to acquire Player Y. In my view, one of the primary causes for Chip Kelly’s downfall was that he possessed authority over both the talent evaluation process and the resource allocation process, but seemed to ignore the questions demanded of the resource allocation process and instead favored his technocratic talent evaluation judgments.

DePodesta’s title – Chief Strategy Officer – is telling. If the Browns hired DePodesta to elevate analytics to an even footing with scouting, or even to largely replace scouting with analytics, they could have easily given him a more traditional title such as General Manager or President of Football Operations.  These titles would have indicated that he is filling a familiar role, but would perform such role differently.  Instead, they gave him a novel title to, I suspect, emphasize that he is performing a new role.   As such, I think this move is less about altering the methods of the talent evaluation process and more about elevating the resource allocation process to a position of supremacy within the organization. I predict that if the Browns experience success during DePodesta’s tenure, the result will not be that technocrats lose their talent evaluation jobs to analytically inclined executives, but rather that analytically inclined executives will step into organizational structures as the bosses of the technocrats. Football lifers should determine who can block and tackle, but hedge fund managers, CFOs, and corporate lawyers should make the final decisions as to asset management and risk allotment.

Bryce Johnston is the creator of Commitment Index and the co-creator of Expected Contract Value. Bryce earned his Juris Doctor, magna cum laude, from Georgetown University Law Center in May 2014, and currently works as a corporate associate in the New York City office of an AmLaw 50 law firm. Before becoming a contributor to overthecap.com, Bryce operated eaglescap.com for 10 NFL offseasons, appearing multiple times on 610 WIP Sports Radio in Philadelphia as an NFL salary cap expert. Bryce can be contacted via e-mail at bryce.l.johnston@gmail.com or via Twitter @NFLCapAnalytics.