To Design and Analyze an Method for Unsupervised Recurrent All-Pairs Optical Flow Field Transform Algorithm

Abstract

The SMURF, a powerful technique for optical flow unsupervised learning that closes the gap with supervised methods, exhibits good cross-dataset generalization, and even allows for "zero-shot" depth estimation. SMURF introduces several important improvements: full-image warping for learning to predict out-of-frame motion, multi-frame self-supervision for improved flow estimates in occluded regions, and most importantly, modifications to the unsupervised losses and data augmentation that allow the RAFT architecture to operate in an unsupervised setting. These developments, in our opinion, take unsupervised optical flow one step closer to becoming truly practical, enabling optical flow models trained on unlabeled videos to deliver accurate pixel-matching in areas where labeled data is lacking.