Holistic Video Understanding is a joint project of the KU Leuven, University of Bonn, KIT, ETH, and the HVU team.
In the last years, we have seen tremendous progress in the capabilities of computer systems to classify video clips taken from the Internet or to analyze human actions in videos. There are lots of works in video recognition field focusing on specific video understanding tasks, such as action recognition, scene understanding, etc. There have been great achievements in such tasks, however, there has not been enough attention toward the holistic video understanding task as a problem to be tackled. Current systems are expert in some specific fields of the general video understanding problem. However, for real-world applications, such as, analyzing multiple concepts of a video for video search engines and media monitoring systems or providing an appropriate definition of the surrounding environment of a humanoid robot, a combination of current state-of-the-art methods should be used. Therefore, in this workshop, we intend to introduce the holistic video understanding as a new challenge for the video understanding efforts. This challenge focuses on the recognition of scenes, objects, actions, attributes, and events in the real world user-generated videos. To be able to address such tasks, we also introduce our new dataset named Holistic Video Understanding~(HVU dataset) that is organized hierarchically in a semantic taxonomy of holistic video understanding. Almost all of the real-world conditioned video datasets are targeting human action or sport recognition. So our new dataset can help the vision community and bring more attention to bring more interesting solutions for holistic video understanding. The workshop is tailored to bringing together ideas around multi-label and multi-task recognition of different semantic concepts in the real world videos. And the research efforts can be tried on our new dataset.
The main objective of the workshop is to establish a video benchmark integrating joint recognition of all the semantic concepts, as a single class label per task is often not sufficient to describe the holistic content of a video. The planned panel discussion with world’s leading experts on this problem will be a fruitful input and source of ideas for all participants. Further, we invite the community to help to extend the HVU dataset that will spur research in video understanding as a comprehensive, multi-faceted problem. We as organizers expect to receive valuable feedback from users and from the community on how to improve the benchmark.
|13:40||Invited Speaker 1:||Juan Carlos Niebles|
|14:10||Invited Speaker 2:||Raquel Urtasun|
|14:40||Coffee and Posters|
|15:25||4 Oral Talks||23, 16, 8, 22|
|16:15||Invited Speaker 3:||Du Tran|
|16:45||Invited Speaker 4:||David Ross|
|17:15||3 Oral Talks:||7, 4, 9|
|17:50||Conclusion and Awards|
4. Recurrent Convolutions for Causal 3D CNNs; Gurkirt Singh, Fabio Cuzzolin
7. Video Representation Learning by Dense Predictive Coding; Tengda Han, Weidi Xie, Andrew Zisserman
8. Towards Segmenting Anything That Moves; Achal D Dave, Pavel Tokmakov, Deva Ramanan
9. Video-Text Compliance: Activity Verification Based on Natural Language Instructions; Mayoore Jaiswal et al.
16. HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips; Antoine Miech et al.
22. Deep Multimodal Feature Encoding for Video Ordering; Vivek Sharma, Makarand Tapaswi, Rainer Stiefelhagen
23. CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning; Rohit Girdhar, Deva Ramanan
-Recurrent Convolutions for Causal 3D CNNs; Gurkirt Singh, Fabio Cuzzolin
-Level Selector Network for Optimizing Accuracy-Specificity Trade-offs; Ahsan Iqbal, Jürgen Gall
-End-to-End Video Captioning; Silvio Olivastri, Gurkirt Singh, Fabio Cuzzolin
-Video Representation Learning by Dense Predictive Coding; Tengda Han, Weidi Xie, Andrew Zisserman
-Towards Segmenting Anything That Moves; Achal D Dave, Pavel Tokmakov, Deva Ramanan
-Video-Text Compliance: Activity Verification Based on Natural Language Instructions; Mayoore Jaiswal, Frank Liu, Anupama Jagannathan, Anne Gattiker, Inseok Hwang, Jinho Lee, Matt Tong, Sahil Dureja, Soham Shah, Peter Hofstee, Valerie Chen, Suvadip Paul, Rogerio Feris
-Interpretable Spatio-temporal Attention for Video Action Recognition; Lili Meng
-Markov Decision Process for Video Generation; Vladyslav Yushchenko, Nikita Araslanov, Stefan Roth
-Next-flow: Hybrid multi-tasking with next-frame prediction to boost optical-flow estimation in the wild; Nima Sedaghat, Mohammadreza Zolfaghari
-Use What You Have: Video retrieval using representations from collaborative experts; Yang Liu, Samuel Albanie, Arsha Nagrani, Andrew Zisserman
-HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips; Antoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, Josef Sivic
-Coupled Recurrent Network (CRN); Lin Sun
-Deep Multimodal Feature Encoding for Video Ordering; Vivek Sharma, Makarand Tapaswi, Rainer Stiefelhagen
-CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning; Rohit Girdhar, Deva Ramanan
-Class-Agnostic Object Tracking with a Focus on the Object; Achal D Dave, Pavel Tokmakov, Cordelia Schmid, Deva Ramanan
-Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Ranking; Alaaeldin M El-Nouby, Shuangfei Zhai, Graham Taylor, Josh Susskind
Prospective authors will be invited to submit a regular paper of previously unpublished work (ICCV paper format) or an extended abstract of a published or ongoing work. The review process will be double blind. All the submissions will be peer-reviewed by the international program committee. Accepted papers will be presented as posters or contributed talks and will be considered non-archival and published via the Open Access versions, provided by the Computer Vision Foundation. Accepted extended abstracts will be presented at the poster session.
*You can submit papers in three different formats:
Facebook AI Research