DEVELOPER BLOG

Creating More Engaging Sports Broadcasts With AI
In order to expand the scope of possibilities for augmented reality applications and player performance tracking in professional sports, researchers at Stats Perform, a sports analytics company, have developed an AI-based method that makes camera calibration faster and more flexible.
Sports teams use vision-based tracking systems to analyze player performance, and broadcasters deploy AR to enhance the viewing experience (e.g. the Virtual 3 in basketball, the First Down Line in the NFL). These solutions depend on high-quality camera calibration and as such are usually constrained to pre-calibrated, fixed cameras or real-time parameter updates from pan-tilt-zoom cameras. More flexible solutions exist but don’t work well in busy, non-uniform environments like a basketball game where the field is crowded and changes appearance from arena to arena.
To overcome these limitations, the researchers devised a novel neural network that combines semantic segmentation, camera pose initialization, and homography refinement into a single network architecture.
Trained with an NVIDIA TITAN RTX GPU, with the cuDNN-accelerated TensorFlow deep learning framework, their method enables them to determine camera homography of a single moving camera using only the camera frame and the sport as inputs.
Their technique also reduced the inference time to 4 milliseconds (a reduction of two orders of magnitude compared to the previous state-of-the-art), making it suitable for live broadcast.
“The evaluation results show that our method outperforms the previous state-of-the-art in challenging scenarios like basketball and achieves competitive performance in relatively static environments like soccer,” the researchers stated in their paper, End-to-End Camera Calibration for Broadcast Videos. The paper will be presented at the Computer Vision and Pattern Recognition (CVPR) conference this week.

According to the researchers a typical sporting event is covered by multiple separate cameras, but today only a handful of them provide the calibration capabilities needed for these applications. Extending these capabilities to more cameras and more sports will improve insights and help sports leagues build a tighter relationship with their fans.