Detail View

Online temporal action localization from an untrimmed video stream is a difficult problem in computer vision. It is challenging because i) in an untrimmed video stream, more than one action instance may appear, including background scenes, and ii) in online settings, only past and current information is available. Therefore, temporal priors, such as the average action duration of training data, which have been exploited by previous action detections methods, are not suitable for this task because of the high intra-class variation in human actions.

We propose a novel online action detection framework that considers actions as a set of temporally ordered subclasses and leverages a future frame generation network to cope with the limited information issue associated with problem outlined above. Additionally, we augment our data by varying the lengths of videos to allow the proposed method to learn about the high intra-class variation in human actions. We evaluate our method using the THUMOS'14 dataset for an online action detection scenario and demonstrate that the performance is comparable to state-of-the-art methods that have been proposed for offline settings.

1 Introduction 1
2 Related Work 5
3 Method 8
3.1 Detecting action candidate spots . . . . . . . . . . . . . . . . . 10
3.2 Learning visual traits regarding a temporal order . . . . . . . . 10
3.3 Generating future frames . . . . . . . . . . . . . . . . . . . . . . 14
3.4 Detecting actions by modeling temporal correlations . . . . . . . 14
3.5 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.6 Data augmentation . . . . . . . . . . . . . . . . . . . . . . . . . 17
4 Experiments 19
4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . 21
5 Conclusion and Future work 29