A Smartphone-Based Pointing Technique in Cross-Device Interaction

Abstract

In this paper, the author develops a novel cross-device cursor position estimation system for transferring a mobile device’s four-direction 2D movement to a cursor’s four-direction movement on a large display device; this is achieved through the use of sound-source localization and machine-learning algorithms. This system is implemented in two steps. First, the system starts the cursor’s position initialization by taking advantage of the theory of soundsource localization. Second, the system transfers the mobile device’s movement to the cursor’s movement by means of a machine-learning model. This newly developed system improves usability of cross-device applications by offering intuitive 2D move gesture and multi-user interaction context and removes physical distance restrictions. A pilot test has been conducted, and the results have demonstrated that naïve Bayes and gradient boosting are suitable for detecting the 2D movement of a mobile device.