Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data

Miroslav Purkrabek, Jiri Matas
Visual Recognition Group, Czech Technical University in Prague
The 18th IEEE International Conference on Automatic Face and Gesture Recognition
Paper      Supplementary      Code      arXiv

Pose estimation trained on COCO (left) and by our method (right). Red frames indicate OKS values below 0.8.
The limbs are: right arm, right leg, left arm and left leg.

Abstract

Methods and datasets for human pose estimation focus predominantly on side- and front-view scenarios. We overcome the limitation by leveraging synthetic data and introduce RePoGen (RarE POses GENerator), an SMPL-based method for generating synthetic humans with comprehensive control over pose and view. Experiments on top-view datasets and a new dataset of real images with diverse poses show that adding the RePoGen data to the COCO dataset outperforms previous approaches to top- and bottom-view pose estimation without harming performance on common views. An ablation study shows that anatomical plausibility, a property prior research focused on, is not a prerequisite for effective performance. The introduced dataset and the corresponding code are available on the project website.

Datasets

We introduce new datasets and annotations available for download here:

  • Improved annotations for seq1 and seq2 of the PoseFES dataset
  • RePo : manually annotated dataset of (rare) poses from extreme views
  • RePoGen : syntehtic dataset of poses from extreme views

Results

Comparison of the SOTA (ViTPose) trained on the COCO dataset and our method trained on the COCO dataset augmented with RePoGen images.
The limbs are: right arm, right leg, left arm and left leg.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

Comparison with the COCO-trained method

First image description.

BibTeX

If you use our work, please cite it as follows: 


@misc{purkrabek2023improving,
  title={Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data}, 
  author={Miroslav Purkrabek and Jiri Matas},
  year={2023},
  eprint={2307.06737},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}