Active Spaces Roy H. Campbell [email protected] Computer Science Department

Active Spaces Roy H. Campbell [email protected] Computer Science Department University of Illinois at UrbanaChampaign Roy H. Campbell ([email protected] .edu)

“Where virtual reality puts people inside a computer-generated world, ubiquitous computing forces the computer to live out here in the world with people. Mark Weiser Roy H. Campbell ([email protected]

Introduction A standard definition for an Active Space does not exist, but it is evolving. There are different approaches regarding functionality required and implementation details. Roy H. Campbell ([email protected]

Overview Georgia Tech, Microsoft: University of Illinois: Emerging perspectives Scale Software Infrastructure Berkeley, M.I.T., Washington: Comparison Roy H. Campbell ([email protected]

Active Spaces Properties Some common requirements: Location Detection Contextual Model Autonomous and Responsive Behavior Correlation of Heterogeneous Input Interaction Mechanism Roy H. Campbell ([email protected]

Example Implementations Classroom 2000. Georgia Tech Goal: Preservation and augmentation of electronic notes taken by students and teachers. Attach and synchronize audio, video and annotations to class notes Roy H. Campbell ([email protected]

Example Implementations Classroom 2000. Georgia Tech The architectural scheme must be able to evolve with time. Scheme phases: Preproduction: Automate tasks to begin class Live Capture: Record and timestamp all events associated to class. Postproduction: Support development of interfaces to integrate and associate related streams. Access: Provide a universal access method (WEB) Roy H. Campbell ([email protected]

Example Implementations Classroom 2000. Georgia Tech First implementation: Useful as an experimentation prototype. Equipment and software not good enough. Lessons learned were applied to the next implementation. Roy H. Campbell ([email protected]

Example Implementations Classroom 2000. Georgia Tech Living Laboratory Completely equipped classroom that hosted several courses Improved software that allowed better media integration. Software infrastructure did not require as much attention from users as in previous implementation (invisibility required by ubiquitous computing) Roy H. Campbell ([email protected]

Example Implementations Classroom 2000. Georgia Tech Their conclusions: There should be a motivating application. Built system should address some notion of scale: Physical space covered. Number of individuals involved. Number and variety of devices supported. Amount of time over which the application is run. System must be subjected to real and everyday usage before being subject of authentic evaluation. Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft Focuses on physical home and work environments. Computing must be as natural as lighting Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft People are the central entities of the Active Space. Main characteristics of their intelligent environment: Self-awareness Casual access Extensibility Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft Self-awareness: “EasyLiving spaces must be aware of their own activity and contents to allow appropriate responses to the movement of people and their requests ” Requirements: Geometry People within space People’s actions and preferences Resources available Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft Casual Access “Users should not be required to go to a special place to interact with the computer. Nor should they be required to wear special devices or markers to have the computer know where they are” The computer is everywhere. Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft Extensibility: “EasyLiving capabilities should grow automatically as new hardware is added” Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft Design issues: Sensing and Modeling: video cameras User interfaces: new approaches, migration Privacy Roy H. Campbell ([email protected]

Example Implementations EasyLiving. Microsoft Design issues: Architecture for extensibility Central Server Room Server Device Room Server Device Roy H. Campbell ([email protected] Device Device

Active Spaces UIUC The future of computing is implicit to sense and affect its surroundings interconnected and mobile nimble and adaptive Roy H. Campbell ([email protected]

Vision: Pervasive Computing Anywhere/anytime collaboration “Real world” environments augmented with information spaces Mobility and ubiquity Applications like design and prototyping Roy H. Campbell ([email protected]

Implicit Computing Features embedded in every day devices mobile and interconnected pervasive throughout the environment coupled to information sources active, modifying their environment Analogy telephone (implicit) telegraph (explicit) Roy H. Campbell ([email protected]

Research Vision Distributed, multimodal collaboration situated and mobile varying resources varying collaboration intensities bandwidth, computing, power, and display capabilities focused, interested, and ambient dynamic reconfiguration and adaptation resources and modalities Roy H. Campbell ([email protected]

Intelligent Spaces Smart rooms Virtual environments local, high-bandwidth mobile collaborators Road warriors smart device prototyping and information space introspection Lone rangers intelligent offices – domiciles of mobile users conference and seminar rooms – multiway collaboration remote, low-bandwidth mobile collaborators Prototyping laboratories smart device stereolithographic fabrication Roy H. Campbell ([email protected]

Research Vision Three software pillars dynamic, interoperable software frameworks multimodal, adaptive networking controls for adaptive management Hardware testbeds smart rooms and virtual environments mobile users and devices device prototyping development infrastructure Roy H. Campbell ([email protected]

Smart Rooms Components intelligent physical objects (“rocks”) X10 automation sensors and controls lighting, power, temperature controls commercial speech tools ambient condition sensors user, object, and location identifiers limited vocabulary voice commands voice acknowledgments tracking cameras distributed collaboration and localization Roy H. Campbell ([email protected] DARPA CPOF

Smart Rooms Components (continued) smart white boards “brain storming” and digital ink capture multiple displays conference LCD displays high-resolution flat panels digital video encoders multiway conferences and digital video MPEG and HDTV multicast multimodal networks infrared, spread spectrum radio, Ethernet, and ATM Roy H. Campbell ([email protected]

Lone Rangers Rationale continuous connectivity local, mobile collaborators “drop in” for smart room collaborations Components wearable computers with wireless networks multimedia workstations/PCs Roy H. Campbell ([email protected]

Road Warriors Rationale wide-area mobile collaborators low bandwidth connectivity Components personal digital assistants (PDAs) palmtop computers Palm Pilot Toshiba Libretto and/or Windows CE systems two-way pagers RIM/Bellsouth or Motorola ReFLEX PageWriter Roy H. Campbell ([email protected]

Virtual Environments Components Continuing EVL collaboration drafting table ImmersaDesks gaze tracking hardware tactile data gloves virtual environment software ImmersaDesk prototypes Walls 30 ft x 9ft 10 x 3 CPUs/displays Roy H. Campbell ([email protected]

Prototyping Laboratory Rationale prototyping of smart space objects physical/virtual interconnection Components stereolithographic prototyping system object scanner Roy H. Campbell ([email protected]

Challenge: Interoperable Component Architecture Scalable Flexible Pervasive Dynamic and adaptive Mobile Visual and aural Roy H. Campbell ([email protected]

Challenge: Quality of Service Spectrum of networks and computers Wealth of multimedia and requirements Dynamic requirements and resources Roy H. Campbell ([email protected]

Challenge: Active Space Analysis and Control Modality transduction Heterogeneous devices Information access metaphors Roy H. Campbell ([email protected]

Definitions Active Spaces. UIUC An "Active Space" (AS) is a framework which models and manages physical environments as well as its contents "Space" implies that a principle use of the AS framework is to manage physical spaces, such as campuses, buildings, meeting rooms, cities, offices, cars, buses, trains, etc Roy H. Campbell ([email protected]

Properties Active Spaces. UIUC "Active" notes that the framework is intended to support reactive and customized behavior. Key properties identified so far for Active Spaces: Export an adaptable interaction model, that allows interacting with the active spaces Provide a contextual model Reflective and autonomous Have an associated physical counterpart. Roy H. Campbell ([email protected]

Properties Active Spaces. UIUC Key properties identified so far for Active Spaces: Contain entities with a well defined behavior that can be dynamically created and modified. Provide a concept of inside and outside. Define protocols to interact with other active spaces. Autonomous, contextual and can trigger events spontaneously Roy H. Campbell ([email protected]

Requirements Active Spaces. UIUC Key Technical Requirements: Dynamic Behavior Reflection Adaptability Component Based Infrastructure Roy H. Campbell ([email protected]

Example Implementations Active Spaces. UIUC ARCHITECTURAL REQUIREMENTS Roy H. Campbell ([email protected]

Implementation Active Spaces. UIUC We provide a generic infrastructure that allows implementing different scenarios. Every particular scenario defines the nature of the entities contained in the AS as well as the behavior of the particular AS. The great diversity of Active Space scenarios makes it impossible to define a single Active Space that covers all semantic requirements. Roy H. Campbell ([email protected]

Future Example of Interaction Smart room configures itself as user enters PDA configures itself for room User OS manages accessible room resources in collaboration with room OS Video transferred from smart room HDTV display to palm top as user leaves User accesses smart room from remote facilities User environment follows user movements Roy H. Campbell ([email protected]

2K User-oriented, network-centric OS built from components Based on reflective distributed objects Targets ubiquitous computing/smart spaces Supports mobility and persistence leveraging CORBA, DCOM, Java RMI services and protocols Embedding legacy applications and systems Roy H. Campbell ([email protected]

Distributed Objects Reflective Dynamic dependency model, programmable reconfiguration Self-Aware naming, location, protocol, and implementation self-monitoring, self-configuring, and dependency aware Standardized Object Request Broker (ORB), OMG, Java RMI, or DCOM Roy H. Campbell ([email protected]

Profile Service Env. Service QoS Roy H. Campbell ([email protected] Naming Service

Components Componentized ORBs: dynamic TAO concurrency control and thread models resource allocation and schedulers Microkernel support for ORBs: Off Corbarized device drivers: video, X10 Corbarized distributed OS: 2K LegORB: 8Kbytes, Palmshell, remote video Proxy server: Palm video filter Persistent store & name server: nameORB Roy H. Campbell ([email protected]

Features Demonstrated Configurable ORBs and reflection Programming with dependency objects Self-awareness in ORBs LegORB – a thin ORB PalmMPEG video – proxies for modality X10 ORBs – ubiquity The 2K environment – mobile users Security UIUC Tiny Sesame Roy H. Campbell ([email protected]

Automatic Configuration Prerequisites Specifies component needs HW resources HW capacity software services required Helps implement WYNIWYG Video client example PC with MPEG decoder card 50% of 200 MHz processor required CORBA video server Roy H. Campbell ([email protected]

Automatic Configuration Dynamic Dependence ComponentConfigurator associated with each relevant componen depends on depends on CLIENTS COMPONENT COMPONENT HOOKS Hooked Components CLIENTS Roy H. Campbell ([email protected]

DynamicTAO: Dynamically Configurable CORBA ORB Reflective ORB architecturally selfaware dynamic component (un)loading Reconfiguration of ORB engine CORBA applications Roy H. Campbell ([email protected]

Configuration Agent Application of dynamic TAO A video distribution network code distribution, group reconfiguration, state inspection Configure graph of network Agents multicast update dynamic TAO dependencies synchronize update Future: keep servers and clients running as permitted by dependencies Roy H. Campbell ([email protected]

Context Monitoring Client Client Server O Monitor R B Storage Server User Query Interface Roy H. Campbell ([email protected] Users Users

PalmPilot Integration in 2K System Bootstrapping System Utilization Environment Service 2 Profile Server 1 3 4 Environment 5 Implementation Repository 6 Camera 2K Camera Device Driver 7 Roy H. Campbell ([email protected]

Streaming Video to Palm Pilot MPEG Stream Video Proxy Compressed Bitmap Stream Palm Pilot lacks processing power to decode MPEG Video proxy transforms MPEG streams reduces frame rate, color depth, size sends compressed bitmaps Roy H. Campbell ([email protected]

QoS Components Trading service QoS method providers and QoS methods transcoders and filters Exported service(s) QoS proxy resource coordinator Resource brokers DSRT and communication objects Roy H. Campbell ([email protected]

2K QoS Framework import export 2 client QoS Method Provider Expor t Trading Service 1 server Discovered service(s) 3 6/11 import 5/12 9 QoS Proxy 4 4 RB RB 13 Comm. object Net condition Net condition RB RB : Resource Broker QoS Proxy 10 Comm. object 8 7 RB RB RB : Resource Broker Roy H. Campbell ([email protected] 7 RB

Security: Active Capabilities Mobile security agents Secure bootstrapping process Active capabilities minimal core security services application specific access control Interoperable security policies CORBA compliant security services and APIs Roy H. Campbell ([email protected]

Policy Classes DAC - Discretionary Access Control NDAC - Non . Double DAC DONDAC, domain-oriented . MAC formed from customized NDAC DSP Device Specific Policies DANDAC, device aware . Roy H. Campbell ([email protected]

Security Components Application Client Application Server Active Capability/Certificates Active Capability/Certificates ORB Stub Stub Dynamic Dynamic Policies Policies BOA BOA Security Security Mechanisms Mechanisms Network NetworkTransport Transport Roy H. Campbell ([email protected]

A Light-Weight Security Mechanism: Tiny UIUC SESAME IDL IDL Interface Interface GSS-API GSS-API Tiny Tiny SESAME SESAME Roy H. Campbell ([email protected]

Comparisons Berkeley M.I.T. Washington Roy H. Campbell ([email protected]

Berkeley: Endeavour Funding: Darpa Exhibition Objective of Program: “Achieve nothing less than radically enhancing human understanding through the use of information technology, by making it dramatically more convenient for people to interact with information, devices, and other people” Roy H. Campbell ([email protected]

Berkeley: Endeavour Research areas: Applications Information Utility Information Devices Design Methodologies Roy H. Campbell ([email protected]

Berkeley: Endeavour Key technologies produced: Integration of MEMS-based sensors, actuators, position locators, communicators with imagers (cameras), displays, and hand-portable and other mobile computing devices into the system architecture of the next century. Support for fluid software. High-speed decision making and education/learning spaces. Design methodologies. Roy H. Campbell ([email protected]

Berkeley: Endeavour Existing infrastructure: Interactive University Project Millennium Tertiary Disk Prototype Wireless Network (GSM base transceiver, REFLEX two-way paging basestation, Lucent WaveLAN) Post-PC infrastructure (IBM Workpads) Roy H. Campbell ([email protected]

Washington: Portolano Funding: DARPA Expedition Objective of Program: Invisible User Interfaces Universal Connectivity Intelligent Services Roy H. Campbell ([email protected]

Washington: Portolano Research areas: Devices: Intentional User Interfaces Low-power architectures/devices/communication Application partitioning for low-power Roy H. Campbell ([email protected]

Washington: Portolano Research areas: Middleware Application development/deployment Data-centric networking Scalable discovery/directory services Roy H. Campbell ([email protected]

Washington: Portolano Research areas: Services Composable, horizontally-layered services Distributed databases and data integration Data mining and database layering Roy H. Campbell ([email protected]

Washington: Portolano Existing infrastructure: Devices used in previous ParcTab experiments. Roy H. Campbell ([email protected]

Washington: Portolano Key technologies produced: Portolano infrastructure. User and usage studies of wide deployment. Ubiquitous large displays Application development environment Creation of a set of office application/device suites Modular sensor system Roy H. Campbell ([email protected]

Washington: Portolano Key technologies produced Location sensor technologies Novel automatic identification technologies Flexible active network infrastructures Sensor fusion Extended information-space management infrastructure Making sense of complex information from distributed information sources Roy H. Campbell ([email protected]

M.I.T.: Oxygen Funding: DARPA Objective of Program: “Do more by doing less” Bring new technologies into people’s lives, not vice versa. Improve human productivity. Offer new technology to everybody not only a fraction of people. Roy H. Campbell ([email protected]

M.I.T.: Oxygen Research areas: Hardware Speech recognition Artificial Inteligence Networks OSes Roy H. Campbell ([email protected]

M.I.T.: Oxygen Key technologies produced: Handy 21: multipurpose software configurable PDA. Enviro 21:Stays attached to the environments around the people. Net 21: Creates secure “collaborative” regions among Oxygen users. Speech understanding Knowledge access Collaboration Automation Customization Roy H. Campbell ([email protected]

Conclusion An Active Space is meaningless without a well defined application. Correlation of heterogeneous input data is required to create a software model of the active space. Adaptation of applications based on the software model of the active space guarantees heterogeneous access to the active space. Roy H. Campbell ([email protected]