Articles
Spatio-Temporal Convolutional Framework for Air-Writing Text Recognition via Finger Motion Analysis
Downloads
Keywords
Air-writing
Spatio-temporal convolution
Finger motion analysis
Handwriting recognition
Article Usage Statistics
Live Metrics
Abstract Views
179
File Views
79
PDF
0
HTML
0
Other
79
Total
258
Article Usage Chart
Abstract
File Views
PDF
HTML
Other
Total
Abstract
Air-writing, or the recognition of handwritten text in mid-air, has emerged as a transformative modality for human-computer interaction, enabling natural, gesture-based text input without physical contact. Recent advances in sensor technologies, machine learning, and spatio-temporal modeling have facilitated robust recognition of finger movements to interpret dynamic handwriting trajectories. This review paper critically analyzes the evolution of air-writing recognition frameworks, emphasizing spatio-temporal convolutional architectures for finger motion analysis. By synthesizing methodologies across wearable systems, depth-sensing devices, and neural network-based approaches, we identify key design principles, performance trade-offs, and limitations inherent to current frameworks. Additionally, we highlight practical implications for real-time applications, adaptive input systems, and multimodal integration. Our synthesis reveals that spatio-temporal convolutional models provide a structured approach to capture both the sequential and spatial nuances of finger motion, offering enhanced recognition accuracy and system adaptability. Notably, wearable and sensor-based systems, as demonstrated in (Amma, 2014), establish foundational benchmarks for contemporary AI-driven approaches in air-writing recognition.
Downloads
Download data is not yet available.
References
1. C. Amma, M. Georgi, and T. Schultz, “Airwriting: A wearable handwriting recognition system,” Pers. Ubiquitous Comput., vol. 18, no. 1, pp. 191–203, 2014.
2. M. Arsalan, A. Santra, K. Bierzynski, and V. Issakov, “Air-writing with sparse network of radars using spatio-temporal learning,” in Proc. 25th Int. Conf. Pattern Recognit., 2021, pp. 8877–8884.
3. M. Chen, G. AlRegib, and B.-H. Juang, “Air-writing recognition–Part I: Modeling and recognition of characters, words, and connecting motions,” IEEE Trans. Human-Mach. Syst., vol. 46, no. 3, pp. 403–413, Jun. 2016.
4. V. Carbune, “Fast multi-language LSTM-based online handwriting recognition,” Int. J. Document Anal. Recognit., vol. 23, pp. 89–102, 2020.
5. Y. Chherawala, P. P. Roy, and M. Cheriet, “Feature set evaluation for offline handwriting recognition systems: Application to the recurrent neural network model,” IEEE Trans. Cybern., vol. 46, no. 12, pp. 2825–2836, Dec. 2016.
6. H. Ding, “A compact CNN-DBLSTM based character model for offline handwriting recognition with Tucker decomposition,” in Proc. 14th IAPR Int. Conf. Document Anal. Recognit., 2017, vol. 1, pp. 507–512.
7. H. Ding, K. Chen, and Q. Huo, “Compressing CNN-DBLSTM models for OCR with teacher-student learning and Tucker decomposition,” Pattern Recognit., vol. 96, 2019, Art. no. 106957.
8. S. Espana-Boquera, M. J. Castro-Bleda, J. Gorbe-Moya, and F. Zamora-Martinez, “Improving offline handwritten text recognition with hybrid HMM/ANN models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 767–779, Apr. 2011.
9. Z. Fu, J. Xu, Z. Zhu, A. X. Liu, and X. Sun, “Writing in the air with WiFi signals for virtual reality devices,” IEEE Trans. Mobile Comput., vol. 18, no. 2, pp. 473–484, Feb. 2019.
10. Y. Huang, X. Liu, X. Zhang, and L. Jin, “A pointing gesture based egocentric interaction system: Dataset, approach and application,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, 2016, pp. 16–23.
11. U.-H. Kim, S.-M. Yoo, and J.-H. Kim, “I-keyboard: Fully imaginary keyboard on touch devices empowered by deep neural decoder,” IEEE Trans. Cybern., vol. 51, no. 9, pp. 4528–4539, Sep. 2021.
12. S. Mohammadi and R. Maleki, “Real-time Kinect-based air-writing system with a novel analytical classifier,” Int. J. Document Anal. Recognit., vol. 22, no. 2, pp. 113–125, 2019.
13. S. Mukherjee, S. A. Ahmed, D. P. Dogra, S. Kar, and P. P. Roy, “Fingertip detection and tracking for recognition of air-writing in videos,” Expert Syst. Appl., vol. 136, pp. 217–229, 2019.
14. A. Schick, D. Morlock, C. Amma, T. Schultz, and R. Stiefelhagen, “Vision-based handwriting recognition for unrestricted text input in mid-air,” in Proc. 14th ACM Int. Conf. Multimodal Interact., 2012, pp. 217–220.
15. M. Sepahvand, F. Abdali-Mohammadi, and F. Mardukhi, “Evolutionary metric-learning-based recognition algorithm for online isolated Persian/Arabic characters, reconstructed using inertial pen signals,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2872–2884, Sep. 2017.
16. D. Tao, X. Lin, L. Jin, and X. Li, “Principal component 2-D long short-term memory for font recognition on single Chinese characters,” IEEE Trans. Cybern., vol. 46, no. 3, pp. 756–765, Mar. 2016.
17. X. Zhang, Z. Ye, L. Jin, Z. Feng, and S. Xu, “A new writing experience: Finger writing in the air using a kinect sensor,” IEEE MultiMedia, vol. 20, no. 4, pp. 85–93, Oct.–Dec. 2013.
18. X.-Y. Zhang, F. Yin, Y.-M. Zhang, C.-L. Liu, and Y. Bengio, “Drawing and recognizing Chinese characters with recurrent neural network,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 849–862, Apr. 2018.
19. D. Tran, H. Wang, L. Torresani, J. Ray, Y. LeCun, and M. Paluri, “A closer look at spatiotemporal convolutions for action recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 6450–6459.
20. I. Z. Yalniz and R. Manmatha, “Dependence models for searching text in document images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 1, pp. 49–63, Jan. 2019.