A shared interactive display (i.e. tabletop) provides a large space for collaborative interaction. However, it lacks a private area for accessing sensitive information. On the other hand, a mobile device offers a variety of modalities for personal applications, but it is limited by a small screen. This paper develops a framework that enables fluid and seamless interaction between a tabletop and a mobile device, and thus integrates the merits of each. More specifically, the framework can continuously track multiple users' actions (e.g., hand movement or gesture) on top of a tabletop, and then automatically generate a unique personal interface on a mobile device for each user. Such an inter-device interaction supplements a collaborative workspace (i.e., a tabletop) with a private space (i.e., a mobile device) having diverse modalities. Without losing generosity, we apply an observer pattern to implement the framework based on the Microsoft PixelSense Technology. This framework hides the details of user tracking and inter-device communication. Thus, interface designers can focus on the development of domain-specific interaction by mapping user actions on a tabletop to commands on a mobile device. We conducted a user study, which compared our approach with a standard tabletop interface. The results are promising and justify the usability of the proposed approach.