If you're looking for an accurate prediction of how the 2023-24 NHL season will go, your first port of call probably shouldn't be a large language model–based chatbot.
There are a few key reasons why we know that.
ChatGPT doesn't know any information from beyond September 2021 — among other things that makes it extremely likely to overrate Philipp Grubauer, who has posted a .891 save percentage in 94 games with the Seattle Kraken after finishing third in Vezina Trophy voting in 2020-21.
This chatbot hasn't spent a single hour grinding tape. Sure, it can regurgitate analysis it's seen elsewhere, but as far as the eye test goes, it's useless.
It will tell you repeatedly that you should not take its hockey analysis seriously.
We know this because when morbid curiosity led us to ask it what it thought would happen in the upcoming season, it was extremely resistant to doing so. Upon first request for a prediction, it stated that it didn't have the information to make one and offered the following advice.
"To get an accurate prediction for the 2023-24 NHL season, you would need access to up-to-date information, including player rosters, team dynamics, and performance trends closer to that time. I recommend checking with reliable sports news sources and utilizing data analytics closer to the start of the season for more accurate predictions."
You can empathize with the bot here.
It sucks to be strong-armed in offering an opinion when you simply don't have the necessary information at your disposal. As Abraham Lincoln is often credited with saying — possibly incorrectly — "It's better to remain silent and be thought a fool than to open your mouth and remove all doubt."
One thing that chatbots and humans have in common is that they can be worn down, though.
Given the prompt "I acknowledge you are missing critical information. Make your best estimate anyway." ChatGPT got closer to committing to a prediction. While it still refused to spit out an outcome for 2023-24, it offered some relatively accurate statements about the state of the league that included:
"Teams with strong rosters and recent success, such as the Tampa Bay Lightning, Colorado Avalanche, and Vegas Golden Knights, are likely to remain top contenders for the Stanley Cup."
"Young players like Connor McDavid, Auston Matthews, and Nathan MacKinnon, who were already among the league's best in 2021, may continue to dominate and be in the running for major awards."
"Highly-touted prospects from the 2021 and 2022 NHL Entry Drafts, such as Owen Power, Matthew Beniers, and Brandt Clarke, could have made their mark in the NHL by 2023-24, potentially improving their respective teams."
While there's some quasi-concrete information there, it did not constitute a proper prediction.
That led to the following exchange that we're going to screenshot for you to confirm its authenticity:
There's a lot to chew on here, but what ChatGPT is describing is quite literally the best-case scenario for a Toronto Maple Leafs season, both in terms of result and high drama.
ChatGPT's predictions about Matthews are particularly intriguing as it sees him becoming the Maple Leafs captain despite the fact John Tavares took that role prior to the 2019-20 season. The bot had seen just two seasons of Tavares wearing the 'C' and determined it wasn't going to last.
It also predicted Matthews would be coming off a 60-goal season, which is an optimistic projection considering he'd never topped 47 in a year when ChatGPT stopped getting new information.
He had just scored 41 in 53 games in 2020-21, but getting 60 over a whole season is tough — and according to the bot's info it was a feat that'd been managed just twice since 1995-96 (Steven Stamkos in 2011-12 and Alex Ovechkin in 2007-08).
Another interesting wrinkle is McDavid's fourth Hart Trophy. ChatGPT only knows about two Hart Trophy wins for McDavid, which means it predicted two wins between 2021-22 and 2023-24. That's reasonable but aggressive.
Now, does this "purely fictional" prediction tell us anything of value? Not really.
It's plausible in the sense that the Maple Leafs and Edmonton Oilers are both likely to make the playoffs and a Stanley Cup Final between them is a possible outcome. It's certainly not the likeliest matchup, but there are more far-fetched picks you could make.
The hardware ChatGPT wants to give out to Matthews and McDavid could also make sense since the former has already showed 60-goal form and the latter is always the default Hart Trophy favourite.
While we're not going to pretend to have a firm grasp on the inner workings of ChatGPT, the likeliest reason for this prediction is the sheer amount of content around the Maple Leafs and their Stanley Cup chances — as well as their drought — plus all the ink spilled about McDavid and his team's championship window.
Whatever the case may be, a Maple Leafs fan base starving for a title now has a well-known backer. Unfortunately for Toronto, that backer is an artificial intelligence with no particular aptitude for hockey, or knowledge of quite literally anything from the last 24 months.