RePoGen

Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data

Visual Recognition Group, Czech Technical University in Prague
The 18th IEEE International Conference on Automatic Face and Gesture Recognition
Winner of the Best Poster Award

Abstract

Methods and datasets for human pose estimation focus predominantly on side- and front-view scenarios. We overcome the limitation by leveraging synthetic data and introduce RePoGen (RarE POses GENerator), an SMPL-based method for generating synthetic humans with comprehensive control over pose and view. Experiments on top-view datasets and a new dataset of real images with diverse poses show that adding the RePoGen data to the COCO dataset outperforms previous approaches to top- and bottom-view pose estimation without harming performance on common views. An ablation study shows that anatomical plausibility, a property prior research focused on, is not a prerequisite for effective performance. The introduced dataset and the corresponding code are available on the project website.

Datasets

We introduce new datasets and annotations available for download here:

Improved annotations for seq1 and seq2 of the PoseFES dataset
RePo : manually annotated dataset of (rare) poses from extreme views
RePoGen : syntehtic dataset of poses from extreme views

Results

Comparison of the SOTA (ViTPose) trained on the COCO dataset and our method trained on the COCO dataset augmented with RePoGen images.
The limbs are: right arm, right leg, left arm and left leg.

First image description.

BibTeX

If you use our work, please cite it as follows:

@INPROCEEDINGS{purkrabek2024improving, author={Purkrabek, Miroslav and Matas, Jiri}, booktitle={2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG)}, title={Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data}, year={2024}, volume={}, number={}, pages={1-9}, keywords={Space vehicles;Training;Three-dimensional displays;Pose estimation;Gesture recognition;Data models;Orbits}, doi={10.1109/FG59268.2024.10582011}}

Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data

Pose estimation trained on COCO (left) and by our method (right). Red frames indicate OKS values below 0.8. The limbs are: right arm, right leg, left arm and left leg.

Abstract

Datasets

Results

First image description.

First image description.

First image description.

First image description.

First image description.

First image description.

First image description.

First image description.

BibTeX

Pose estimation trained on COCO (left) and by our method (right). Red frames indicate OKS values below 0.8.
The limbs are: right arm, right leg, left arm and left leg.