PalmPad: Typing on Your Hand with a Single RGB Camera

A team from Tsinghua University has developed Palmpad, a novel mixed reality interaction technology. Using only an RGB camera and an Arduino, Palmpad projects a virtual keyboard onto the user's palm. The user types by tapping virtual keys with their index finger from the opposite hand. The system overcomes previous limitations of unreliable fingertip tracking and poor touch detection through sophisticated video processing and a CNN for accurate fingertip positioning. An Arduino UNO detects finger-palm contact using impedance changes, achieving 97% touch recognition accuracy. Palmpad offers a convenient and innovative approach to mixed reality interaction.
Read more