An inside look at the statistic platform the selection committee will use to pick the Football Four
The College Football Playoff executive in charge of developing one of the key tools for his selection committee found his solution from a Tweet.
As the Playoff administrators assembled their group of 13 selectors during the last year-and-a-half, chief operating officer Michael Kelly knew he had to find a tool to keep the assemblage of college football luminaries informed.
The BCS computers were out. The polls would be of no use. No one wanted the rigid tools similar to the ones used by the basketball committee — RPI, strength of schedule and so on.
The Playoff executives wanted the selection committee to be the last word, and handing the selection committee opponent records or total offense and total defense wouldn’t suffice.
Lucky for Kelly, a Twitter follower stepped up.
Ex-college baseball players, brothers Stephen and Scott Prather and a third partner Drew Borland, once had aspirations of starting a data-driven coach search firm that leaned heavily on an extensive database they developed as a side project. (Stephen Prather and Borland both played at Vanderbilt from 1998-2000; Scott Prather played at Georgia Tech from 1996-98 and spent five years in the minor leagues for the Cardinals.)
They had trouble catching on in the search firm market, but athletic departments and coaches liked their database, dubbed Coaches By The Numbers. They went forward with an analytic platform called SportSource Analytics, culling play-by-play and season data from college football games going back to 2001.
A year-and-a-half ago, with the Coaches By The Numbers consulting business in full swing, Stephen Prather noticed Kelly’s conundrum, and he thought SportSource Analytics might be the solution.
"Data will play a part. Gut will be a part. Film will be a part. That's the way it should be."
The Playoff executive committee and SportSource Analytics team (which also came to include Marty Couvillon, proprietor of cfbstats.com) met several times over the course of 18 months, including at CFP headquarters in Dallas with the selection committee.
“That Twitter (interaction) led to an online demo of our product,” Prather said. “Over the next year-and-a-half it went from ‘this is pretty cool’ to ‘can we build something specifically for the committee.’”
Kelly and the selection committee needed a tool that would provide the committee a wealth of comparative data, from surface-level statistics to more in-depth metrics. The interface had to be simple enough for even the more tech-adverse members of the committee. And the committee members had to be able to access it at anytime, anywhere.
“We found this to be the most user-friendly and what we needed for our committee,” Kelly said. “What we liked was that there are hundreds and hundreds of categories of raw data, but also the ability to compare that to a certain number of teams. They even have great ways to go deeper.”
The College Football Playoff signed SportSource Analytics to a two-year contract to provide an exclusive platform for the selection committee. The team will be available through the selection process to provide tech support and answer questions about the tool, but both parties are clear that SportSource Analytics will not influence the committee on selection.
The platform will contain raw data on a per-play, per-possession, per-game and season-long basis but not a stand-alone metric similar to an RPI or Sagarin rating.
Prather and Kelly both said avoiding a “magic bullet” statistic was key. If the committee members can’t explain their reasoning, the data wouldn’t be useful, Prather said.
“We have nothing to do with the decision,” said Stephen Prather, who is a vice president for a commercial real estate company in Nashville. “We are building tools for them to look at data. ... We’re trying to give you ways of looking at data. We’re not trying to tell you what to look at.”
So what will the selection committee be able to access through the SportSource Analytics tool? That depends on the committee member.
The tool will allow committee members to compare teams in more than 60 statistical categories from the basic statistics — total offense and defense, turnover margin — but also more advanced metrics including yards per play, points per possession and detailed red zone success metrics.
The platform also will allow for detailed strength-of-schedule breakdowns including combined record of opponents, record of opponents’ opponents, record of conference opponents, records against ranked teams and teams with winning records.
Committee members also will be able to compare team performance in certain games, i.e. statistical data in games against winning teams. The platform will provide team sheets with data on all 128 teams, including detailed schedule analysis, statistical ranks and how they compare to the nationwide average.
The platform will allow committee members to dive as deep as they’d like, allowing them to customize more than 100 different rankings: How many points per possession did a team score against conference teams with winning records? That’s available.
How often are defenses holding top-25 opponents to three-and-outs? That’s available.
Which team has played the teams with the best cumulative conference record? That is available, too.
The tool also will be adaptive by request of selection committee members, so SportSource Analytics can add or create stat categories on demand.
“What we liked was that there are hundreds and hundreds of categories of raw data, but also the ability to compare that to a certain number of teams,” Kelly said.
Of course, there’s the possibility committee members won’t take a deep statistical stat dive, either.
That’s not going to hurt Prather’s feelings. For him, maybe the playoff spots shouldn’t be determined exclusively by red zone defense.
“Data will play a part. Gut will be a part. Film will be a part,” Prather said. “That’s the way it should be.”