2017-098 – OpenPose – Realtime Multiperson 2D Keypoint Detection from Video

OpenPose has represented the first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images. It is authored by Ginés Hidalgo, Zhe Cao, Tomas Simon, Shih-En Wei, Yaadhav Raaj, Hanbyul Joo, and Yaser Sheikh. It is maintained by Ginés Hidalgo and Yaadhav Raaj. OpenPose would not be possible without the CMU Panoptic Studio dataset. We would also like to thank all the people who have helped OpenPose in any way.Library main functionality: Multi-person 15, 18 or 25-keypoint body/foot keypoint estimation, including 6 foot keypoints. Runtime invariant to number of detected people. Multi-person 2×21-keypoint hand keypoint estimation. Multi-person 70-keypoint face keypoint estimation. 3D real-time single-person keypoint detection. Calibration toolbox: Estimation of distortion, intrinsic, and extrinsic camera parameters. Single-person tracking for further speedup or visual smoothing. Input: Image, video, webcam, Flir/Point Grey, IP camera, and support to add your own custom input source (e.g., depth camera). Output: Basic image + keypoint display/saving (PNG, JPG, AVI, …), keypoint saving (JSON, XML, YML, …), keypoints as array class, and support to add your own custom output code (e.g., some fancy UI). OS: Ubuntu (20, 18, 16, 14), Windows (10, 8), Mac OSX, Nvidia TX2. Hardware compatibility: CUDA (Nvidia GPU), OpenCL (AMD GPU), and non-GPU (CPU-only) versions. Command-line demo for built-in functionality. C++ API and Python API for custom functionality. E.g., adding your custom inputs, pre-processing, post-posprocessing, and output steps. CMU CTTEC CTTEC-Flintbox@andrew.cmu.edu 412-268-7393