Smart glasses to assist disabled and elderly in everyday tasks

Almost everyone knows at least one of the following situations: you want to bake a cake but cannot remember the exact recipe, you do not know how to use a high-tech kitchen stove to prepare a delicious meal or simply how to repair a bicycle. In such everyday situations it may be helpful to receive unobtrusive and intuitive support from an adaptive technical system that operates along in a largely unnoticed and restriction-free manner. This project focuses on the development and testing of intelligent glasses.

ADAMAAS provides situation, user, and action specific feedback in a textual, visual or avatar based format on a transparent virtual plane superimposed on user?s field of view.

It combines techniques from memory research, eye tracking and vital parameter measurement (such as pulse or heart rate), object and action recognition (Computer Vision), as well as Augmented Reality (AR) with modern diagnostics and corrective intervention techniques. The system will be able to identify problems in actual action processes, to react when mistakes are made, as well as to display situation and context dependent assistance in textual, visual or avatar based format superimposed on a transparent virtual plane in users' field of view. ADAMAAS aims to provide support for people to be able to live a self-sufficient life in an age appropriate way according to their mental and physical capabilities. Thus, the system will be able to suggest new action options and to induce selective learning processes. ADAMAAS is therefore not a stationary, but a mobile assistance system.

In order to allow for age appropriate, individual assistance in everyday situations, cognitive action sequences will be diagnosed in a first step using the structural dimensional analysis motoric method (SDA-M; Schack, 2012). The method applied is already well established in cognitive psychology and captures the mental representation structures using a hierarchical splitting procedure (Fig. 2). The system is then able to identify user specific problems in the execution of actual action sequences. Based on this information, the actual visual focus and the detected objects in the environment, the system is able to provide adequate feedback, i.e. by displaying situation and user specific help comments and hints.

A particular problem in the context of technical assistance systems is the handling of sensitive data (e.g., the recording and storing of it). Therefore, we will pay particular attention to data protection and data security. The recorded data shall be transferred via a secured data connection on a local host system, which is installed on-site. Therefore, no displaced person can access or distribute this potentially sensitive data.

Several research groups based in Bielefeld University and external companies will work together to achieve the project goals. The v. Bodelschwingh Foundation Bethel will contribute their expertise in diagnosis and action assistance, as well as taking care of disabled and elderly people. Furthermore, they will provide an ideal test environment and provide us with access to different target groups for the system evaluation stage of the project. Bielefeld University is represented by three working groups: Sport Science, the Center of Excellence CITEC (Cognitive Interaction Technology) and the Faculty of Technology. Together they bring a wealth of experience in the areas of Image Processing/Computer Vision, Machine Learning, mental representation structures for diagnostics and action execution, Feedback-Prompting, Sport and Movement science, Augmented Reality (AR), artificial systems, character animation, Man-Machine Interaction, User Experience, ELSI (ethical, social and legal implications), and finally speech processing. The Eye tracking company SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH (SMI), based in Berlin, will provide the necessary hardware (see Fig. 1) and Velamed GmbH Science in Motion (Medizintechnik und Biomechanische Konzepte), based in Cologne, integrates the sensors for measuring the vital parameters. The iTIME (Institute for Technological Innovation, Market Development and Entrepreneurship) provide advice in economic matters, to conduct market analysis and the preparation of a marketing concept. The initiative Bielefeld 2000plus - Research Projects for the Region will establish a working group on the topic "Intelligent Assistance Systems for Elderly People" to foster the dialog between academia and representative from society.The Hettich Holding GmbH & Co. oHG will evaluate the AR-Eyetracking Glasses in the context of the acquisition and support in a complex assembly process.

Basic principles of the ADAMAAS system: Detection of the cognitive action structure (left). Supportive prompting on the AR plane (right).

For realizing diagnostic and action assistance we draw on Bethels comprehensive experience in the field of disabled and elderly persons care. The v. Bodelschwingh Foundation Bethel are involved in diagnostic processes and assistance of action in a real world context (workshop and living unit). They also contribute to the system evaluation for people with special needs (elderly and handicapped people).

All in all, the aim of this project is to get from a stationary status diagnostic system to a mobile and dynamical -adaptive action support and monitoring system which is able to react to failures and to provide individualized prompting feedback for action support.

http://www.uni-bielefeld.de/sport/arbeitsbereiche/ab_ii/research/adamaas.html