Temporal Shift Module for Efficient Video Understanding

TSM is an efficient and light-weight operator for video recognition [on edge devices]. [...] Conventional methods using 3D convolution for temporal modeling are computationally expensive, making it difficult to be deployed on embedded devices which have a tight power constraint. In this ICCV’19 paper, we propose Temporal Shift Module (TSM) that can achieve the performance of 3D CNN but maintain 2D CNN’s complexity by shifting the channels along the temporal dimension. TSM enables real-time low-latency online video recognition and video object detection. [...] On NVIDIA Jetson Nano, it achieves a low latency of 13ms (76fps) for online video recognition.