SmarTrack - Multi-person Tracker

SmarTrack is a multi-camera person tracking system. It computes the ground location of people utilizing a coarse shape-plus-color signature, and is designed to work effectively in multi-person scenario where frequent and persistent occlusions occur among the persons. How it works in brief:

• For detection, a ground occupancy map is generated using motion features extracted from multiple views. The modes of the map represent those ground locations that most likely explain the image motion under a 3D human shape hypothesis. In a verification step, every mode of the occupancy map is tested for consistency of shape-model projection and extracted image motion: if confirmed, a new track is instantiated and a colour descriptor is extracted from shape-model projections to form a 3D shape-plus-appearance model of the new target.
• For tracking, a particle filter updates ground location hypotheses using these 3D shape-plus-appearance models. The course-of-dimension induced by appearance dependencies (notably, occlusions) leading to exponential complexity in multi-target tracking is hereby overcome: predictions are updated with a joint occlusion-aware shape-model projection under a quadratic complexity upper bound, leading to a scalable solution to multi-person tracking.

A more detailed project description is available here. Recent research objectives are described here.

Patents: US7965867 EP2302589 US8436913 EP1879149

Key Publications:

• O. Lanz: Approximate Bayesian Multibody Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 28(9):1436-1449, 2006.
• O. Lanz, S. Messelodi: A Sampling Algorithm for Occlusion Robust Multi Target Detection. IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2009.
• T. Hu, S. Messelodi, O. Lanz: Dynamic Task Decomposition for Decentralized Object Tracking in Complex Scenes. Computer Vision and Image Understanding (CVIU), 134():89-104, 2015.

Related Publications:

• J. Varadarajan, R. Subramanian, S. Rota Bulò, N. Ahuja, O. Lanz and E. Ricci: Joint Estimation of Human Pose and Conversational Groups from Social Scenes. International Journal of Computer Vision (IJCV), 2017.
• X. Alameda-Pineda, J. Staiano, R. Subramanian, L. M. Batrinca,  E. Ricci, B. Lepri, O. Lanz, N. Sebe: SALSA: A Novel Dataset for Multimodal Group Behavior Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 38(8):1707-1720, 2016.
• Y. Yan, E. Ricci, G. Liu, O. Lanz and N. Sebe: A Multi-task Learning Framework for Head Pose Estimation under Target Motion, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 38(6):1070-1083, 2016.
• E. Ricci, J. Varadarajan, R. Subramanian, S. Rota Bulò, N. Ahuja and O. Lanz, "Uncovering Interactions and Interactors: Joint Estimation of Head, Body Orientation and F-formations from Surveillance Videos", International Conference on Computer Vision (ICCV), 2015.
• X. Alameda-Pineda, Y. Yan, E. Ricci, O. Lanz and N. Sebe: Analyzing Free-standing Conversational Groups: a Multimodal Approach. ACM International Conference on Multimedia (ACM MM), 2015.
• A.K. Rajagopal, R. Subramanian, E. Ricci, R.L. Vieriu, O. Lanz, R. Kalpathi and N. Sebe: Exploring Transfer Learning Approaches for Head Pose Classification from Multi-view Surveillance Images. International Journal of Computer Vision, 109(1-2):146-167, 2014.