Advanced Stats Season Review Part 4: Evaluating Goals and Predictions

After looking at the big picture and then doing a deep dive on the offense and defense, I want to take this week to evaluate how the 2020 season compared with predictions we made here and whether GT met the particular goals we set for the season.

Win Total

Georgia Tech ended up at 3-7. This was on the low-end of fan expectations coming into the season. But we can look more closely to measure this performance against preseason advanced stats expectations, game week expectations, and the actual underlying performance during the game. Were three wins more than expected, less than expected, or about right?

We can begin answering these questions by using Bill Connelly’s SP+ numbers, closing Vegas spreads on each game, and a win expectancy metric that I have developed incorporating success rate, yards per play, and EPA/play margin in each game.

Preseason SP+ ratings predicted a team that would win just about 4 games, while the ratings difference between GT and its opponents the week of each game predicted just under a 4 win season. Vegas was slightly lower on GT than SP+ and saw just under 3.5 wins according to each week’s closing line. Finally, the underlying predictive metrics of success rate, yards per play, and EPA/play saw a team that played at just over a 3.5 win level. As you can see, all of these numbers exceed our actual win total of 3 wins.

That’s both bad news and good news. The bad news is that we underperformed expectations on the field, which could point to deployment and tactical weakness on the coaching staff. The good news is that on the scoreboard, we also underperformed our own underlying metrics, which is a stat that strongly tends to regress towards the mean year over year. In other words, we are not likely to continue to win less games in coming years than our underlying metrics suggest. Bill Connelly’s win expectation number had an even larger discrepancy, pegging GT for 1.3 wins more than actual based on his win expectancy numbers.

Bottom line: GT missed expectations on the scoreboard, but exceeded them in the underlying stats that are more predictive of future performance.

Season Goals

In a couple of my offseason preview articles last summer, I identified a few stats to pay particular attention to for tracking GT’s progress from 2019 to 2020. From the summer:

“What should you pay particular attention to in 2020 to see if things are moving in the right direction?

• The Passing success rate needs to come in around 40%, which would put us in the middle of the pack nationally and demonstrate significant progress in the quarterback room.
• The Power Success Rate should improve to around 70%, which again would put us in the middle of the pack and give evidence to the upgrades we need on the offensive line.

Where should we look for improvement in 2020 if the defense is going to be able to carry the team to greater success?

• The power success rate allowed by the defense needs to be closer to 70%. This will mean the defense getting off the field far more often and will create more chances for the offense with better field position.
• The havoc rate needs to tick up a couple of percentage points to somewhere around the national average of 21%, to get the opposing offenses off schedule more. This will help cover over some of the weaknesses up front. Look for more creative and aggressive pressure packages from the edge and outside linebacking spots.”

How’d we do?

As you can see above, we did not exceed a single one of the goals I set. We did, however, improve in four of the categories from 2019, the only exception being defensive havoc rate, which is a significant concern going forward as we detailed last week.

What ongoing issues does this point to? Offensive and defensive line struggles manifest in the power success rate on both sides of the ball. The offensive passing success rate improved dramatically, but we still lack a reliable, go-to receiver and need more consistency from the quarterback position. The defensive run stuff rate reflects on the talent upgrades still needed at DT and LB, and the abysmal defensive havoc rate continues to point to a lack of top end talent throughout the defense. The coaches seemed particularly motivated to rectify this last issue with a number of the transfers they brought in this off season.

Bottom line: We got better in 2020, but the talent level, especially on both lines of scrimmage, still is not where it needs to be.

Season Predictions

I’m going to use this space to hold myself accountable as well. I went back through a number of the predictions I offered leading up to the season, and I need to self-evaluate.

In our final season preview post before the first game, I offered a game by game breakdown of what I expected.

We had a pretty good feel for this team going into the 2020 season. I was too optimistic about being able to pull out close wins against BC and NCSU; the Syracuse game defied all predictive capability, but otherwise we were pretty spot on.

I also offered a few more specific predictions in that post:

Offensive MVP: Tucker Gleason — Very Wrong

Defensive MVP: Juanyeh Thomas — Very Wrong

Best Moment: Winning in Tallahassee with a True Freshman at Quarterback - CORRECT

Worst Moment: Losing to Pitt, derailing hopes for an improbably special season and reminding us that we still have building to do - Correctish (minus the derailing hopes part)

Areas of Focus

In my most controversial article to date, I suggested that GT fans try not to worry about the win loss record in 2020 and offered a few other ideas to gauge progress. How did we do there?

1. Do we have clear direction at the quarterback position entering 2021?

Resounding Yes

2. Do we make clear progress in efficiency?

Yes on offense, no on defense

3. Does 2021 recruiting hit the same level as 2020?

No if we count only high school seniors, but yes if we include the transfers.

Conclusion

Are we happy with 3-7? No.

Did we make progress in 2020? Yes.

Is there reason to think 2021 will be better than 2020? Yes.

212 days until kickoff.