Seeing from Magic Mirror: Contrastive Learning from Reconstruction for Pose-based Gait Recognition

Shibei Meng, Saihui Hou, Yang Fu, Xuecai Hu, Junzhou Huang, and Yongzhen Huang

[Paper] [Dataset] [Code]

• StreamGait: A large-scale, unlabelled gait dataset collected from 353 YouTube live-stream channels across 30 countries / 81 cities.
• MirrorGait: A self-supervised 3D-aware pre-training framework that lifts 2D poses to 3D for contrastive learning, achieving SOTA on Gait3D, GREW and OUMVLP-Pose with minimal fine-tuning.

From FastPoseGait to GPGait++: Bridging the Past and Future for Pose-based Gait Recognition

Shibei Meng^*, Yang Fu^*, Saihui Hou, Xuecai Hu, Chunshui Cao, Xu Liu, and Yongzhen Huang

[Paper] [Code]

• FastPoseGait: We developed the FastPoseGait open-source toolbox and used it to establish a fairer, more consistent benchmark for gait recognition algorithms.
• GPGait++: We proposed GPGait++, a new method that significantly outperforms all prior pose-based work in challenging cross-domain scenarios.

GPGait: Generalized Pose-based Gait Recognition

Yang Fu^*, Shibei Meng^*, Saihui Hou, Xuecai Hu, and Yongzhen Huang

[Paper] [Code]

We introduce GPGait, a framework designed to improve the generalization of pose-based gait recognition across different datasets. To achieve this, it first uses a Human-Oriented Transformation (HOT) to create a unified and feature-rich pose representation. Then, it employs a Part-Aware Graph Convolutional Network (PAGCN) to effectively extract local and global spatial features from the keypoints, making the model more robust and effective in diverse scenarios.

OmniDiff: A Comprehensive Benchmark for Fine-grained Image Difference Captioning

Yuan Liu, Saihui Hou, Saijie Hou, Jiabao Du, Shibei Meng, and Yongzhen Huang

[Paper]

• OmniDiff: We construct OmniDiff, a high-quality dataset comprising 324 diverse scenarios, encompassing both real-world complex environments and 3D synthetic settings.
• M³Diff: We integrate a plug-and-play Multi-scale Differential Perception (MDP) Module into the MLLM architecture, establishing a strong base model specifically tailored for IDC.

Cut out the Middleman: Revisiting Pose-based Gait Recognition

Yang Fu, Saihui Hou, Shibei Meng, Xuecai Hu, Chunshui Cao, Xu Liu, and Yongzhen Huang

[Paper] [Code]

We introduce a new method for gait recognition using heatmaps directly from upstream methods as the primary representation, instead of the more traditional skeletons. It also identifies the key challenges associated with using heatmaps and provides a simple yet non-trivial solution to overcome them.