Skip to main content

HandDGCL: Two-hand 3D reconstruction based disturbing graph contrastive learning.

Han, B., Yao, C., Wang, X., Chang, J. and Ban, X., 2023. HandDGCL: Two-hand 3D reconstruction based disturbing graph contrastive learning. Computer Animation and Virtual Worlds, 34 (3-4), e2186.

Full text available as:

[img]
Preview
PDF
CASA2023_hanbing_v0.7.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

8MB

DOI: 10.1002/cav.2186

Abstract

Virtual reality (VR) and augmented reality (AR) applications are becoming increasingly prevalent. However, constructing realistic 3D hands, especially when two hands are interacting, from a single RGB image remains a major challenge due to severe mutual occlusion and the enormous diversity of hand poses. In this article, we propose a disturbing graph contrastive learning strategy for two-hand 3D reconstruction. This involves a graph disturbance network designed to generate graph feature pairs to enhance the consistency of the two-hand pose features. A contrastive learning module leverages high-quality generative features for a strong feature expression. We further propose a similarity distinguish method to divide positive and negative features for accelerating the model convergence. Additionally, a multi-term loss is designed to balance the relation among the hand pose, the visual scale and the viewpoint position. Our model has achieved state-of-the-art results in the InterHand2.6M benchmark. Ablation studies show the model's great ability to correct unreasonable hand movements. In subjective assessments, our graph disturbance learning method significantly improves the construction of realistic 3D hands, especially when two hands are interacting.

Item Type:Article
ISSN:1546-4261
Uncontrolled Keywords:graph contrastive learning; hand pose estimation; hand shape reconstruction
Group:Faculty of Media & Communication
ID Code:38863
Deposited By: Symplectic RT2
Deposited On:10 Aug 2023 10:21
Last Modified:01 Jun 2024 01:08

Downloads

Downloads per month over past year

More statistics for this item...
Repository Staff Only -