Towards 3D Human Posture Estimation Using Multiple Kinects Despite Self-Contacts

Geodesic distance graph edges: a) Naive (left, note the invalid edge near Worker 1’s feet) b) MVS refined (right)

Abstract

We present a marker-less human motion capture system that uses multiple RGB-D cameras to estimate the 3D posture of multiple people online at interactive rates in an indoor workspace measuring approximately 5m × 5m × 2m. An interesting aspect of this work is how we handle the self-contact problem. We propose a novel multi-view voting scheme (MVS) to fuse measurements from different 2D or 3D algorithms. As a proof of concept, we present an MVS implementation that fuses optical flow images from each view and labels points in the current instance using the previously estimated posture. These labels allow us to trim invalid edges in a geodesic distance graph model and improve localization of geodesic extrema corresponding to the head, hands and feet for posture estimation. The system performs at ∼8.3 Hz with a cumulative latency of ∼570.40 ms and a projected median localization error of ∼0.149 m. In addition, we propose a new multi-view Kinect and Vicon publicly accessible motion capture dataset for validation and benchmarks.

Files

• phan2015towards.pdf - article
• 20140610-HumanTracking.mp4 - video demo of the human tracking
• 20140610-CHARM.mp4 - video demo of the CHARM project human robot interaction

Dataset Links

• http://www.cim.mcgill.ca/~apl/database - McGill-Reparti Artificial Perception Database
• http://www.cim.mcgill.ca/~apl/database/sadb - Situational Awareness Database (SADB) server and client library. See "Datasets" link on left toolbar or jump to direct link for download of multiple Kinect and Vicon sequences stored as SADB dumps.

Bibtex

@InProceedings{phan2015towards,
  Title       = {Towards 3D Human Posture Estimation Using Multiple Kinects Despite Self-Contacts},
  Author      = {Phan, Andrew and Ferrie, Frank P.},
  Booktitle   = {15th IAPR International Conference on Machine Vision Applications},
  Pages       = {567--571},
  Year        = {2015}
}