On Beyond GUI: Wearable, Heterogeneous, Adaptive Steven Feiner

On Beyond GUI: Wearable, Heterogeneous, Adaptive Steven Feiner Computer Graphics & User Interfaces Lab Department of Computer Science Columbia University New York, NY 10027 Supported in part by AFRL, NSF, ONR, and gifts from Alias Systems, IBM, Mitsubishi Electric Research Labs, and Microsoft CHI 2006 Workshop: What is the Next Generation of Human-Computer Interaction? Montreal, Canada April 23, 2006 1

Wearable Corners of watch bezel serve as “tactile landmarks” to guide user’s finger Cursorless Eyes-free G. Blaskó and S, Feiner. An interaction system for watch computers using tactile guidance and bidirectional segmented strokes. Proc. ISWC 2004. 2

Wearable Cursorless Eyes-free G. Blaskó and S, Feiner. An interaction system for watch computers using tactile guidance and bidirectional segmented strokes. Proc. ISWC 2004. 3

Wearable Cursorless Eyes-free G. Blaskó and S, Feiner. An interaction system for watch computers using tactile guidance and bidirectional segmented strokes. Proc. ISWC 2004. 4

Wearable Cursorless Glanceable G. Blaskó, Blaskó C. Narayanaswami, and S, Feiner. Prototyping Retractable String-Based Interaction Techniques for Dual-Display Mobile Devices. Proc. CHI 2006. 5

Heterogeneous Coordinated use of different displays and interaction devices Multiple displays Projected desktop Handheld High-res monitor See-through headworn Multiple interaction devices Multi-touch table Tracked hands/heads Speech H. Benko, E. Ishak, Ishak and S, Feiner. Collaborative mixed reality visualization of an archaeological excavation. Proc. ISMAR 2004. 6

Heterogeneous Coordinated use of different displays and interaction devices Multiple displays Projected desktop Handheld High-res monitor See-through headworn Multiple interaction devices Multi-touch table Tracked hands/heads Speech H. Benko, E. Ishak, Ishak and S, Feiner. Collaborative mixed reality visualization of an archaeological excavation. Proc. ISMAR 2004. 7

Heterogeneous Coordinated use of dimensionalities “Pull” 2D image into 3D model “Push” 3D model into 2D image H. Benko, E. Ishak, Ishak and S, Feiner. Cross-dimensional gestural interaction techniques for hybrid immersive environments. Proc. VR 2005. 8

Heterogeneous Coordinated use of dimensionalities “Pull” 2D image into 3D model “Push” 3D model into 2D image H. Benko, E. Ishak, Ishak and S, Feiner. Cross-dimensional gestural interaction techniques for hybrid immersive environments. Proc. VR 2005. 9

Adaptive Change user interface in response to environment Modify layout of virtual objects to avoid overlap in AR B. Bell, S. Feiner, & T. Höllerer. öllerer View management for virtual and augmented reality. Proc. UIST 2001. 10

Adaptive Change user interface in response to environment Modify transparency to reveal important content on desktop E. Ishak, Ishak and S, Feiner. Interacting with hidden content using content-aware free-space transparency. Proc. UIST 2004. 11

Three Themes for Future UIs Wearable Cursorless Eyes free, glanceable, Heterogeneous Mix of users, displays, devices, dimensionalities, Adaptive Dynamically modified in response to changes in environment, tasks, 12