Articles
Robust Spatio-Temporal Framework for Skeleton-Based Human Action Recognition Across Varying Viewpoints
Downloads
Keywords
Skeleton-based action recognition
Spatio-temporal framework
View-invariant representation
Human motion modeling
Article Usage Statistics
Live Metrics
Abstract Views
216
File Views
78
PDF
0
HTML
0
Other
78
Total
294
Article Usage Chart
Abstract
File Views
PDF
HTML
Other
Total
Abstract
Human action recognition (HAR) has emerged as a critical research domain in computer vision, particularly for applications in surveillance, healthcare, and human-computer interaction. Skeleton-based representations offer a compact, informative abstraction of human motion while mitigating challenges posed by background clutter and illumination changes. However, existing skeleton-based HAR approaches often struggle with viewpoint variations and temporal dynamics, limiting generalization across diverse real-world scenarios. This paper proposes a robust spatio-temporal framework for skeleton-based action recognition that integrates joint-level spatial features with temporally encoded motion descriptors to achieve view-invariant recognition. The framework leverages hierarchical modeling of skeleton sequences, incorporating temporal patterns while preserving spatial structural information. Comprehensive analysis on benchmark skeleton datasets demonstrates that the proposed method outperforms baseline models in accuracy and robustness under varying viewpoints. The study further provides theoretical justification for the chosen feature representation and evaluates the limitations and potential avenues for future development.
Downloads
Download data is not yet available.
References
1. J. K. Aggarwal and M. S. Ryoo, “Human activity analysis: A review,” ACM Comput. Surv., vol. 43, no. 3, pp. 16:1–16:43, Apr. 2011.
2. A. Jalal, S. Kamal, and D. Kim, “A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments,” Sensors, vol. 14, no. 7, pp. 11735–11759, Jul. 2014.
3. Jalal, J. T. Kim, and T. S. Kim, “Human activity recognition using the labeled depth body parts information of depth silhouettes,” in Proc. 6th Int. Symp. Sustain. Healthy Buildings, vol. 27, 2012, pp. 1–8.
4. Jalal and S. Kamal, “Real-time life logging via a depth silhouette-based human activity recognition system for smart home services,” in Proc. 11th IEEE Int. Conf. Adv. Video Signal Based Surveillance, 2014, pp. 74–80.
5. H. Y. Cheng and J. N. Hwang, “Integrated video object tracking with applications in trajectory-based event detection,” J. Vis. Commun. Image Representation, vol. 22, no. 7, pp. 673–685, Oct. 2011.
6. H. Chen, G. Wang, J. H. Xue, and L. He, “A novel hierarchical framework for human action recognition,” Pattern Recognit., vol. 55, pp. 148–159, Jul. 2016.
7. D. D. Dawn and S. H. Shaikh, “A comprehensive survey of human action recognition with Spatio-temporal interest point (STIP) detector,” Vis. Comput., vol. 32, pp. 289–306, 2016.
8. E. Cippitelli, S. Gasparrini, E. Gambi, and S. Spinsante, “A human activity recognition system using skeleton data from RGBD sensors,” Comput. Intell. Neurosci., vol. 2016, pp. 1–14, 2016, Art. no. 4351435.
9. M. Ding and G. Fan, “Multilayer joint gait-pose manifolds for human gait motion modeling,” IEEE Trans. Cybern., vol. 45, no. 11, pp. 2413–2424, Nov. 2015.
10. Y. Du, Y. Fu, and L. Wang, “Skeleton based action recognition with convolutional neural network,” in Proc. 3rd IAPR Asian Conf. Pattern Recognit., 2015, pp. 579–583.
11. J. Frey and D. Dueck, “Clustering by passing messages between data points,” Science, vol. 315, no. 5814, pp. 972–976, 2007.
12. F. Lv and R. Nevatia, “Recognition and segmentation of 3D human action using HMM and Multi-class adaboost,” in Proc. Eur. Conf. Comput. Vis., 2006, pp. 359–372.
13. M. E. Hussein, M. Torki, M. A. Gowayyed, and M. El-Saban, “Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations,” in Proc. 23rd Int. Joint Conf. Artif. Intell., 2013, pp. 2466–2472.
14. N. Junejo, E. Dexter, I. Laptev, and P. Perez, “View-independent action recognition from temporal self-similarities,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 172–185, Jan. 2011.
15. A. A. Liu, W. Z. Nie, Y. T. Su, L. Ma, T. Hao, and Z. X. Yang, “Coupled hidden conditional random fields for RGB-D human action recognition,” Signal Process., vol. 112, pp. 74–82, 2015.
16. Y. Yang, S. Iyengar, P. Kuryloski, and R. Jafari, “Distributed segmentation and classification of human actions using a wearable motion sensor network,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2008, pp. 1–8.
17. M. Jiang, J. Kong, G. Bebis, and H. Huo, “Informative joints based human action recognition using skeleton contexts,” Signal Process. Image Commun., vol. 33, pp. 29–40, 2015.
18. Yu and F. Yun, “Human action recognition and prediction: A survey,” Int. J. Comput. Vis., vol. 130, pp. 1366–1401, 2022.
19. Xia, C. C. Chen, and J. Aggarwal, “View invariant human action recognition using histograms of 3D joints,” in Proc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2012, pp. 20–27.
20. F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy, “Sequence of the most informative joints (SMIJ): A new representation for human skeletal action recognition,” J. Vis. Commun. Image Representation, vol. 25, no. 1, pp. 24–38, 2014.
21. R. Poppe, “A survey on vision-based human action recognition,” Image Vis. Comput., vol. 28, no. 6, pp. 976–990, Jun. 2010.
22. H. Rahmani, A. Mahmood, D. Q. Huynh, and A. Mian, “HOPC: Histogram of oriented principal components of 3D point clouds for action recognition,” in Proc. Eur. Conf. Comput. Vis., 2014, pp. 742–757.
23. P. Wang, W. Li, C. Li, and Y. Hou, “Action recognition based on joint trajectory maps with convolutional neural networks,” Knowl. based Syst., vol. 158, pp. 43–53, 2018.
24. Warchol and T. Kapuscinski, “Human action recognition using bone pair descriptor and distance descriptor,” Symmetry, vol. 12, no. 1580, pp. 1–12, 2020.
25. Weinland, R. Ronfard, and E. Boyer, “A survey of vision-based methods for action representation, segmentation and recognition,” Comput. Vis. Image Understanding, vol. 115, no. 2, pp. 224–241, Feb. 2011.
26. X. Yang and Y. Tian, “Effective 3D action recognition using Eigen joints,” J. Vis. Commun. Image Representation, vol. 25, no. 1, pp. 2–11, 2014.
27. X. Yang and Y. Tian, “Eigen joints-based action recognition using Naive–Bayes-nearest-neighbor,” in Proc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2012, pp. 14–19.
28. R. Vemulapalli, F. Arrate, and R. Chellappa, “Human action recognition by representing 3D skeletons as points in a lie group,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 588–595.
29. Y. Hsu, C. Liu, T. Chen, and L. Fu, “Online view-invariant human action recognition using RGB-D Spatio-temporal matrix,” Pattern Recognit., vol. 60, pp. 215–226, 2016.
30. J. K. Aggarwal and M. S. Ryoo, “Human action analysis: A review,” ACM Comput. Surv., vol. 43, no. 3, pp. 16:1–16:43, Apr. 2011.