Interfaces and Interaction techniques

Before working on mobile and connected devices, a main interest of our team was 3D interaction, and especially in virtual reality setups with large displays. Even if the difference between the sizes of the the displays is huge (30 m2 vs 30 cm2), we believe that part of our experience based on interaction in immersive setups can be adapted to MCD. Starting from the creation of the Iparla project, our researches focus mainely on mobile HCI.

  Previous work with large displays
  Mobile HCI

PREVIOUS WORK WITH LARGE DISPLAYS

Large-displays, used as collective visualization interfaces, allow several co-located participants to be immersed in virtual environments (VE). In spite of their potential for group works, large-displays are often under-used because the users can not interact easily and efficiently with the visualized VE. Consequently, we worked on new techniques for interaction with VE displayed on large screens.

CAT : Control Action Table

Participants: Martin Hachet, Patrick Reuter, Florence Tyndiuk, Pascal Guitton.

The CAT is a 6 DOF freestanding input device designed for interaction with 3D data displayed on large screens. The CAT has been developed to favor a non-constraining, sensitive and fast interaction for groups applications. With the CAT, the rotations (cyclic) are directly controlled by an isotonic sensing mode while the translations (infinite) are controlled by an isometric sensing mode. This allows the direct and easy manipulation of 3D objects, or the navigation in wide virtual environments. Moreover, a tablet that is fixed on the tabletop can be used for accurate 2D interaction techniques or for the control of the system, while keeping immersed in 3D environments.

A user study has shown that the CAT is more efficient than a standard 6 DOF rate controller for 3D manipulation tasks, and that the user preferences are in its favor. The CAT has been used for applications of project review and interactive theater [HG03]. This new input device favors the development of real applications using VE visualized on large-displays. [Hac03, HDG03, HRG03, HGRT03, HGRT04,]

The CAT has been presented at SIGGRAPH 2005 and is now available as a commercial product (see Immersion).

Video

Manipulation of a 3D object
2D and 3D interaction

Real time gesture recognition

Participants: Jean-Baptiste de la Rivière, Pascal Guitton.

We are considering the goal to film one's hand or body in front of a large display in order to recover its posture. This data will then be interpreted according to a predefined interaction technique. While most of such systems rely on appearance-based approaches, we have chosen to investigate how far a model-based one could be efficient. The real-time algorithm we are developing modifies a 3D hand or body model pose in order for its projection to match the input silhouette. Other characteristics include the integration of constraints both on parameters values and parameters evaluation order. Occlusions don't lead the system to systematically fail, but adding cameras will help to disambiguate the current model pose. Great tracking speeds were reached [DG03] [DG03a] [HDG03] [DG04] . Future improvements include the integration of classical image analysis techniques and development of specific interaction techniques.

Real and virtual hand mapping

Collaborative interaction

Participants: Martin Hachet, Yoshifumi Kitamura.

We work in collaboration with the the Human Interface Engineering Lab (University of Osaka), for the development 3D interfaces for collaborative and immersive interaction. The first interface we developed aims at enhancing interaction with Illusionhole, which is a collaborative visualization interface. By rotating a circular interface, the users are able to easily manipulate virtual objects, as if they were rotating real ones. The second interface consists in using PDA as remote controls for interaction with 3D environments displayed on large screens [HK05]. In such settings, numerous users can interact from their own device. We use the new camera-based bi-manual interface we describe bellow for interaction from the PDA.

Interaction with IllusionHole

Mouse emulation using laser

Participants: Jean-Baptiste de la Rivière, Pascal Guitton.

We used 3 video cameras to track a laser dot on the large screen, where an application is displayed. The 2D positions of the laser spot onto the display were sent to the application running, and processed according to specific interaction techniques. Thus, our system allows users to directly interact with the application thanks to their laser pointer, without the need of an external operator who manipulates the model.

Interaction from laser pointers

Study of human factors involved

Participants: Florence Tyndiuk, Christophe Schlick, Gwenola Thomas.

The main object of this work is to understand the differences in human performances during travel and manipulation tasks in Virtual Reality. The better understanding of how subjects use virtual reality interfaces can help to construct interfaces more adapted to cognitive users' processes. We have settled a double experimentation that we have performed on one hundred subjects. For each subject, we measure a set of cognitive factors as attention and motor skills and their performances in two virtual reality tasks (manipulation and travel) with two distinct screens (large and small) and the CAT. After analysis of the statistical data, the major result dealing with the screen size influence have appeared. Subjects with lower spatial abilities improve their performances with larger screens whereas the others do not show significant changes when they use distinct screens. With some small adaptations, the experiment can be performed for an analysis of interactions on small devices.

MOBILE HCI

Bi-manual interaction from video tracking

Participants: Martin Hachet, Joachim Pouderoux, Pasacl Guitton, Jean-Christophe Gonzato.

Recent advances in mobile computing allow the users to deal with 3D interactive graphics on handheld computers. Although the computing resources and screen resolutions grow steadily, user interfaces for handheld computers do not change significantly. Consequently, we designed a new 3 degrees of freedom interface adapted to the characteristics of handheld computers [HPG05]. This interface, tracks the movement of a target that the user holds behind the screen by analyzing the video stream of the handheld computer camera. Consequently, the users benefit from bi-manual interaction. The position of the target is directly inferred from the color-codes that are printed on it using an efficient algorithm. The users can easily interact in real-time in a mobile setting. The visualization of the data is good as the target does not occlude the screen and the interaction techniques are not dependent on the orientation of the handheld computer. We used the interface in several test applications for the visualization of large images such as maps, the manipulation of 3D models, and the navigation in 3D scenes. This new interface favors the development of 2D and 3D interactive applications on handheld computers. An experiment allowed us to demonstrate that our interface was faster than a classical stylus interface for a 2D searching task. Moreover, the user preferences were in favor of our new bi-manual interface [HPGG05].

Bi-manual interaction