The 'Coach's Eye' v Analytics
There have been studies showing that, while coaches are confident that they have perfect memory of competition events (paraphrase), they actually do not. They have confirmation bias, recency bias, etc etc. (Here is one.)
So, we know that if a coach only uses 'the eye test' it is flawed. What this implies is that you can't trust a coach's recall of a game, but you CAN trust analytics. We can imply this because analytics were used as the gold standard in the study.
I was listening to a Bill Simmons podcast with Kevin Durant, and they talked a lot about 'watching games'. Specifically, Durant was adamant that 'Blog Boys should watch games not just read analytics.'
So my question is, if COACH OBSERVATION ONLY is not as accurate recalling analytics as just doing analytics, are ANALYTICS ONLY as accurate at recalling games actions as observed by coaches?
In my opinion, using ANALYTICS ONLY would also be a flawed way of recalling the game. But I'm a coach, so of course I would believe this.
I think we can assume that only using analytics would miss parts of the game that only watching would pick up. So my question is, what parts would it miss? Perhaps this would be worth a research paper? I suspect it would miss things like:
- Decision-Making. A coach can see what an athlete is trying to do, regardless of what the outcome is. They might be happy with it, despite a poor outcome, or visa versa.
- Injury Limitations. A coach may know a player has an injury and so is happy with their performance, despite that, and be happy to keep them in the team while they recover from injury. Analytics will only see that their performance has dropped off.
- Interpersonal Interactions. There may be conflict in the team which the coach needs to deal with by playing one player over another, even if individual performance isn't as good.
- Bad Officiating. The coach may favourably remember a play that was ruled against them because they knew it was a bad call. Analytics won't pick this up.
- etc etc etc.
I think you are spot-on Alexis. The 'real' situation of the game - player interactions, game situation, referee's influence etc. are vital for assessing the 'standard of performance' for individuals and the team overall. One goal scored in a vital part of the game appears the same as any other goal in an analytical sense but it is a game changer in terms of the game outcome.
ReplyDeleteThanks for your feedback. But please note, I think all the game is real. The analytics and the coach observations!
DeleteI am an analyst, and from my perspective, I would prefer the insights of a coach to mere analytics every time. Analytics picks up only what some programmer/analyst has taught the system/algorithm to see, and only a coach/analyst combination (such as yourself, perhaps) can hope to match the insights of a good coach - always with an air of scepticism, of course. However, analytics can be consistent (consistently wrong?).
ReplyDeleteCouldn't agree more Chris. Thanks for your thoughts.
ReplyDelete