22 November 2021 Multi-stream mixed graph convolutional networks for skeleton-based action recognition
Boyuan Zhuang, Jun Kong, Min Jiang, Tianshan Liu
Author Affiliations +
Abstract

The skeleton-based action recognition task has recently drawn much attention in computer vision, and graph convolutional networks (GCNs) have shown great advantages in this task. However, in shallow layers of GCNs, the sharing of convolution kernels in regular graph convolution (RGC) may result in the underutilization of some convolution kernels due to the particularity of the skeleton data. To capture richer locally discriminative action features, we propose the exclusive kernel graph convolution (EKGC) for the shallow layers of GCNs, which assigns the vertexes exclusive convolution kernels for graph convolution. At the same time, following the mixed graph convolution strategy, RGC is retained in the deep layers of the network, and the mixed graph convolutional network (MGCN) is proposed to utilize EKGC and RGC reasonably in all layers of the network. In addition, as a new form of skeleton information in GCN-based methods, angle information is explored in our work. Compared with joint data and bone data, angle data have better invariance against rotation and scaling, and they expand the relationships between skeletons and enrich the data representation of action. Finally, our final model, the multi-stream MGCN, is validated on two challenging datasets, achieving the state-of-the-art performance on the NTU-RGBD dataset and making competitive performance on a kinetics-skeleton dataset.

© 2021 SPIE and IS&T 1017-9909/2021/$28.00© 2021 SPIE and IS&T
Boyuan Zhuang, Jun Kong, Min Jiang, and Tianshan Liu "Multi-stream mixed graph convolutional networks for skeleton-based action recognition," Journal of Electronic Imaging 30(6), 063012 (22 November 2021). https://doi.org/10.1117/1.JEI.30.6.063012
Received: 9 September 2021; Accepted: 8 November 2021; Published: 22 November 2021
Lens.org Logo
CITATIONS
Cited by 2 scholarly publications.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Convolution

Bone

Data modeling

Performance modeling

Motion models

Spine

3D modeling

RELATED CONTENT


Back to Top