Air Writing App with Real-Time Hand Tracking and Handwriting Recognition using Fingertip Detection and Drawing

Abstract

This research explores the development of an innovative Air Writing Application utilizing real-time hand tracking and handwriting recognition. Central to this study is the integration of fingertip detection and virtual drawing to enable seamless user interaction on a virtual canvas. Leveraging cutting-edge technologies like OpenCV and MediaPipe, the system employs real-time video frame processing to detect hand gestures and interpret handwriting using Euclidean distance-based recognition. The methodology incorporates advanced preprocessing techniques, including auto-orientation, auto-contrast adjustment, and grayscale conversion, ensuring robust recognition accuracy. Deep learning models trained on curated datasets were evaluated to optimize handwriting recognition, with results demonstrating significant improvements through data augmentation. The system’s real-time responsiveness and intuitive interface enhance Human-Computer Interaction (HCI), addressing limitations of earlier static and dynamic handwriting recognition methods. Predictive analyses suggest promising user adaptability and multimodal integration for future applications. This work contributes to the evolving field of HCI, providing insights into gesture recognition’s role in handwriting applications and setting the foundation for future advancements in air writing technologies. Key ethical considerations, including user confidentiality and informed consent, were meticulously addressed throughout the study.