TRUST: ROBOTICS AND HAPTICS FOR EXTREME INTERACTION AND UNIVERSAL DESIGN

22 02 2009

by BRIAN DUFFY, LIZBETH GOODMAN, JEREMI SUDOL, JAMES BROSNAN, JANA RIEDEL

This paper chronicles the past five years of research and practice engaged in by the Trust Project Team, most recently working on site at the Stephen Hawking School for children and young people with complex multiple disabilities. TRUST seeks to offer young people with limited physical ability the tools with which to engage in rich 3d imaginative environments that encourage relaxation and healing. The project has to date been tested in various iterations in the Montefiore Hospital for Children (Bronx, New York), The CRC (Central Remedial Clinic Dublin), the KK Hospital for Children (with NTU Gamelab, Singapore) and the Stephen Hawking School, London. The haptic elements of the game and interaction design have been employed to great effect in the most recent iteration of our InterFACES Project as well. The results have been featured in the Science Museum (London) showcase on Future Games, and will soon be redesigned for more site specific installations.

BACKGROUND TO THE PROJECT

In recent years, children’s hospitals worldwide have investigated using aspects of video games and other media tools available to young patients to address such problems as isolation, stress management, and rehabilitation. Online communities represent a novel solution to the problems of isolation, for example, for children in hospital care. The majority of such network interactions are generally text-based, utilising chat and forum clients such as StarBright Web and the Cystic Fibrosis Teams Initiative. There is a growing body of evidence suggesting that these communities, despite being limited to text interfaces, can improve pain scores and may improve depressive symptoms, reduce anxiety, and raise self-esteem.

While the use of interactive digital entertainment, or video games, in hospital settings is not new, studies performed to test more graphical interfaces show a similar potential for gaming networks. The work of Bers et al. examined the use of a three-dimensional graphical virtual city, Zora, which provided the setting for a pilot group of hemodialysis patients to socialize in the same network space and to interact via their avatars . This and similar work demonstrates the potential of virtual gaming as a means to escape from difficulties afforded by their situation in hospital settings. The work of Hoffman et al. also details the use of virtual environments in reducing pain scores when employed as an adjunct to pharmacotherapy for patients with burn and dental pain.

TRUST PROJECT OBJECTIVES & TEAM METHOD

We set out to address the medical and also the social needs of young people in hospital, who seek entertaining games but who do not benefit from over-stimulation or from competitive energies while in the process of learning, or healing. Our co-opetition game model, wherein two players are encouraged to cooperate rather than compete to share a jointly earned goal, has thus been employed at the heart of the TRUST Project. Over the years of project development, we have also come to recognize two distinct levels of objectives in the project: at the higher order level, we seek as in all SMARTlab projects to improve social conditions for groups of disenfranchised people (or, in the most over -simplified and generic description: to make a better world), whereas at the project outcome level linked to delivery, we seek to create empowering games for young people in hospital or with disabilities: games that help to heal, to connect to society and to others in similar situations, and that encourage imaginative free play even for those whose bodies can not move.1

In the first iterations of the game story, play begins with two main characters, Hope and Trust. Each moves in a 3d world but when they move together, they see more of that world and engage in more fulfilling actions and movements through the game space. The story has been customized in each installation, with children participating in the SMARTlab workshops on site in schools and hospitals, so that each child has a major input into the look, feel and action of the storyworld. Lizbeth Goodman leads the workshops, using a methodology based in her Theatre Games training2 as applied to the free play of ‘parts of the memory with pleasure in them’. Storytelling, puppetry, and role play games are employed over periods of weeks or months as appropriate, until each group of children has helped to see their ideas for better more active worlds rendered in image, whether 2d or 3d. And then an Active Chair is built by Brian Duffy, sensors are installed by Jeremi Sudol, and a full range of interactive tests is designed and implemented by the team (differently for each site). The entire process for each site is videoed and documented by Jana Reidel et al.

 

THE BBC INPUT: FROM INTERACTIVE LOUNGE AS INTERFACE TO MINDTOUCH

SMARTlab’s earliest incarnation some fifteen years ago was an emergent new media research group led by scholars of Theatre working in close collaboration with BBC Open University Producers and Technologhy experts. Lizbeth Goodman first worked with Huw Williams (currently head of Research for the BBC) in that early innovation forum. Years later, the BBC collaborative strand is just as strong, and several projects have been undertaken in close collaboration with Dr Marc Price, Senior Engineer/Researcher at BBC R&D, where he specializes in screen systems for audience empowerment. As part of his research and our ongoing collaboration with the BBC, Marc worked with us on site in London, Dublin and Singapore in 2004-5. In 2006, he brought us a new collaborative project in Somatics: using Tai Chi as a physical game interface wherein the ‘meditator is the interface’. This project will run from SMARTlab and selected schools and community centres in East London from 2006-9. That project, now known as MindTouch, is currently gathering first phase results through workshops, and will in time be modified to reach out to young people with disabilities and difficulty in learning and speaking, through TRUST.

MINDTOUCH is an interactive technology project run through SMARTlab and funded by BBC R&D (with the formal project title ‘Embodiment Theory and User Interfaces’), exploring the uses of mobile technologies and biosensors, considered in the contexts of somatics and informatics, in assessing levels of concentration, calm and consternation in the movement and thought patterns of dancers and tai chi practitioners. The aim of the project is to explore the possibilities of ‘the mediator at interface’. Co-PIs Dr Marc Price, Senior Engineer BBC R&D & Professor Lizbeth Goodman, with Camille Baker, Dr Susan Kozel, Dr Li Zhang, Jeremi Sudol et al. Trinity College workshops for the project in 2007 facilitated by Chrissie Poulter in collaboration with the Trinity Department of Computing and Engineering.3

The BBC’s input into the TRUST project has thus been considerable, firstly in the Interactive TV Lounge as an early test environment allowing people in hospital to engage with others in shared virtual environments, and later with the development and testing of biosensor feedback to measure rates of stress and calm in the use of our games and movement exercises.

The ACTIVE CHAIR

The main haptic interface for the game is the Active Chair, invented by Brian Duffy and developed through several iterations of the Trust Project. This has been documented in our previous publications (see below) and is now entering a phase of user testing for interactive display outside the hospital and school settings.

UNIVERSAL DESIGN

Beyond the higher order social design model described here and documented on the Trust Project site, the main objective of the TRUST Project is to develop game-based interactive play in order to aid in children rehabilitation and ease the stresses associated with hospital scenarios. The play environment is designed to be inclusive, i.e. not solely for able-bodied and able-minded people. The virtual environments and game scenarios have been tailored to an audience of 8 to 13 year old children with varying degrees of abilities. TRUST seeks to offer more than distraction, but also a tool for collaboration and connection from the isolating environment of the hospital, and a tool for studying the impact of cooperative game-play on physical, emotional and rehabilitation (http://www.give-trust.org).

To address these objectives, the TRUST framework has integrated several modes of artistic and technological innovation; a game engine and associated software framework and a pneumatically operated, active chair which provides a closed loop haptic interface that is synchronised and congruent with the game play. In addition, an array of joysticks allows users with constrained mobility to interact with the system. The following sections outline the technological innovation that has been developed, as installed and tested at the NTU Gamelab and KK Women & Children’s Hospital in Singapore, 2004=5, and then as redesigned and installed at the Stephen Hawking School, East London, 2006-7.

HOPE

It is no accident, of course, that the second major character in the TRUST Game is called Hope. Each of the two main characters as originally conceived are designed as simulacra for the quality that they lack: Trust moves forward and back on wheels, quickly and freely, but s/he is not able to trust the world around and so often comes crashing to a halt. Hope springs up and down, optimistic but always unable to hope too much, so always caught again in a downward spin. The basic idea in movement is to encourage two players to move the avatars together, to touch hands of the creatures so that they move in tandem and thus equal each other out in the play space, and help each other to see the fuller 3d world.

In closing, it is important to mention the HOPE Project (Hospital-based Online Persistent Paediatric Environment) which first commissioned the team to invent the first version of the bespoke TRUST game for young people in hospital, and which runs to date as a network of medical doctors using gaming in teaching and rehabilitation. Dr Arun Matthews and his team decided that it was important to make healing games for kids, as part of their major study into how young people’s experiences of hospital care have been affected by the social shift towards provision of video games rather than books to hospital wards. Some of our early work with theatre games, dance and sport as triggers for games has evolved from this same shared mission: to encourage healthy active collaboration between characters on and through the screen, and in the ‘real’ spaces of healing and social engagement as well.

The Trust and Hope Projects now run in tandem, each informing the other, as it should be.

See www.hopeconnectskids.org, http://www.hopeconnectskids.org/hopeconspirators.html)

www.give-trust.net

www.smartlab.uk.com

Acknowledgements:

The TRUST Project team would like to thank the Stephen Hawking School and Science Museum London, and also NESTA (the National Foundation for Science Technology and Arts) for the financial support offered for the latest project phase in 2006-7, and BBC R&D, the GameLab at Nanyang Technical University, Singapore, and also the Central Remedial Clinic & Cetre for Independent Living, Dublin. The Carl Sagan Trust and Children’s Health Fund (courtesy of Dr Jeb Weisman of the Harlme Children’s Zone) and the Motion Capture/Movement Lab at NYU funded the first phases of this research in 2002-4, with assistance form Lego Europe and private donors. Thanks also to Dr Arun Mathews (HOPE project lead, Johns Hopkins Medical School) for help with background research.

Footnotes:

1 Goodman recently gave a presentation for Futurelab where she discussed the higher order design principles alongside the user-centred product design outcomes of TRUSAT: Designing for Social Justice think tank, Bristol, October 11-12, 2007.

2 See the full discussion of this training and its impact on the interactive works of SMARTlab in Goodman, L. (2007). Performing in the Wishing Tense: SMARTlab’s Evolution on Stage, Online and in the Sand’. New Theatre Quarterly (Cambridge University Press. XXXI (4), 352-375.

3 For full details see http://www.smartlab.uk.com/2projects/mindtouch.htm

References: 

Battles, H, Wiener, L. (2002). “STARBRIGHT World: Effects of an electronic network on the social environment of children with life-threatening illnesses”. Children’s Health Care, 31(1), 47-68.

Duffy, B.R., Mur Guerri, A., Bourke, J. “Vicarious Adrenaline”, IEEE SMC UK-RI 3rd Workshop on Intelligent Cybernetic Systems (ICS’04), 7- 8 September, 2004 .

Goodman, L. ‘Virtual Interactive Puppetry’, in Leonardo, (MIT Press), Autumn 2001.

Goodman, L with S. Kuppers. ‘Virtual Interactive Puppetry’, in Leonardo, (MIT Press), Autumn 2001.

Goodman, L & Ken Perlin. ‘The Butterfly Project’, in Ruth Aylett and Cañamero, L. eds. Animating Expressive Characters for Social Interactions (Imperial College London, 2002), pp. 43-44 with interactive DVD.

Goodman, L. Creativity and Innovation: Ways forward for the European Union in Cross-Sector and Interdisciplinary International Partnerships. (EC final report for the RADICAL project, 2002).

Goodman, L, et al. ‘SPIRITLEVEL: Making & Using ‘SMART’ tools integrating intelligent systems & performance technologies to connect and empower creative spirits in shared and distant spaces’, in Assistive Technology: Shaping the Future, eds G. Craddock et (Amsterdam, IOS Press, 2003), pp. 89-97.

Goodman, L, with Brian Duffy et al. Anima Obscura. IBC Amsterdam, 2005.

Goodman, L. with K. Milton, eds. A Guide to Good Practice in New Media Content and Tools Creation (by and for artists in the cultural sector). Published online by HEFCE/AHDS in 2003, print edition following best practice citation for this volume published by King’s College London, Office for Humanities Communications Publications number 18, 2005.

Goodman, L. (2007). Performing in the Wishing Tense: SMARTlab’s Evolution on Stage, Online and in the Sand’. New Theatre Quarterly (Cambridge University Press. XXXI (4), 352-375.

Hazzard, A., Celano, M., Collins, M. Markov, Y. (2002). “Effects of STARBRIGHT World on knowledge, social support, and coping of hospitalized children with sickle cell disease and asthma”. Children’s Health Care, 31(1), 69-86.

Holden, G., Bearison, D., Rode, D., Rosenberg, G., and Fishman, M. (1999). “Evaluating the effects of a virtual environment (STARBRIGHT World) with hospitalized children”. Research on Social Work Practice, 9, 365-82

Bers MU, Gonzalez-Heydrich J, DeMaso D.R., (2002). “Use of a computer-based application in a pediatric hemodialysis unit: A pilot study.” Jour Am Acad Child Adolesc Psychiatry,

Hoffman HG, Patterson DR, Carrougher GJ, Sharar SR. (2001) “Effectiveness of virtual reality-based pain control with multiple treatments”. Clinical Journal of Pain, Sep. 17(3):229-35.

Hoffman HG, Garcia-Palacios A, Patterson DR, Jensen M, Furness T 3rd, Ammons WF Jr. (2001). “The effectiveness of virtual reality for dental pain control: a case study”. Cyberpsychol Behaviour 4 (4),527-35. Aug

Price, M. (2004). “The Media Lounge: A Software Platform for Streamed 3D Interactive Mixed Media”, Proc. Int. Symposium on Smart Graphics,

Sudol, J, B.R. Duffy, L. Goodman, J. Brosnan, J. Riedel, "PLAYBOX: A Flexible Framework for All", VR and Motor Disorders, 8th International Conference on Virtual Reality, April 27, 2006, Laval, France. A dvd of the Playbox toolkit was produced upon request for BAFTA Interactive in 2005; copies also available upon request via jana@smartlab.uk.com

Advertisements

Actions

Information

One response

27 02 2009
moko

thank for information….

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: