2 03 2009


At a NASA sponsored conference on Human Systems I reminded the audience of the impact that a photo of the earthrise over a moonscape had on the perception of our planetary condition. The making of this image marked an important moment in the history of human experience. I suggested that a similar event may mark the first voyage to Mars when the blue planet fades into the background of stars before the red one becomes prominent. The sense of profound isolation may not be pleasant but should make for an interesting moment of reflection. One similar to, but I expect orders of magnitude greater than, when mariners first ventured out of sight of land. These experiences have value to our culture in that they shape our understanding of ourselves. Much of what can be learned about extreme environments will be in the form of data, measurements that we can compare to others that we have made in order to shape an understanding of the new in relation to the known. Some have suggested that extreme environments such as those found in extraterrestrial, undersea or polar environments require interrogation by robotic and remote sensing techniques rather than by human exploration and habitation. While these techniques are capable of providing representations that can be understood intellectually, they are incapable of providing a direct experience. Others argue that human beings are the most robust and versatile autonomous control systems available and must be included on missions for that reason. But beyond functionality and instrumentality, arguments that will be continuously eroded by technological innovation in any case, I argue for the irreplaceability of human presence in extreme environments on the grounds of human experience.

However, there is a contradiction here. Extreme environments, as noted by Louis Bec (2007) , do not exist a priori but depend upon the relationship between an environment and the organism in question. We count those as extreme that are hostile to life and are able to venture into them only by virtue of our technological interventions. We participate to the extent that we can remain within a protective technological bubble. These technologies reduce or eliminate the experience of the extreme conditions even as they protect the organism from it. But, can technologies be developed to open extreme environments to experience rather than shielding us from them? I believe that prototype devices have already been developed that show how this can be accomplished. Perceptual prostheses of the kind described here will enable the direct perception of hostile conditions from with in the technological womb. While humans are physiologically capable of experiencing many salient features of their terrestrial environment, this may not be the case for extreme and alien environments. These environments may require the immediate awareness of other spectra or conditions by means of technologically mediated perception. Prosthetic perception may become a key enabling technology for the habitation of extreme conditions in addition to providing the principle justification for a human presence in them.

In order to design perceptual prosthetics, one must have an understanding of the way in which perception develops out of the relationship between an organism and its environment and the constraints that this places on their interaction. Beginning in the 1960s, Paul Bach-y-Rita and colleagues (2000) developed a variety of interfaces that enable the blind to have the perception of spatially located distal objects through the surface of their skin. This is done by mapping the output of a video camera onto a matrix of electrotactile or vibrotactile transducers on the back, thigh, abdomen, forehead or tongue depending upon the application. In these studies two necessary conditions were found. The individual must have volitional movement that affects the output of the sensor and must learn to use the device over an extended period of time. When these conditions were met there came a “spectacular ability to recognize forms…accompanied by an exteriorization of the percepts, which become objects located in space” (Lenay 1997). Sensory substitution, as the technique was called, challenged accepted notions of the senses that were based on an information-processing paradigm and provided evidence for an alternative skill-based theory of perception.

According to this new view, perception is a working knowledge of sensori-motor contingencies. This is an understanding of the ways in which sensory flows are transformed with movement. (O’Regan and Noë 2001). From the standpoint of the organism, perception of self and world resides in the patterns of transformation and stasis within the afferent and efferent sensory fluxes. Sensory substitution devices that preserve a structured relation between these sensory dynamics are able to be integrated with and become part of the perceptual processes of the animal. Although Bach-y-Rita worked in rehabilitation medicine and had as a primary concern devices that ameliorated the effects of missing or faulty sensory modalities, as early as 1972, he suggested that his sensory substitution work could be extended to include a variety of other sensing technologies. However, no examples of sensory substitution devices intended to extend the range of human capabilities can be found. I believe that a structured investigation of these techniques will result in a body of knowledge supporting the incorporation of a wide range of transducers into the human sensorium. This can be accomplished by means of technological devices that facilitate a structured relation between the output of the interface and volitional movements conditioned on the opportunity to develop these skills over time. The ability to design such devices would facilitate the human experience of alien and extreme environments. It will also open human experience to a wider range of conditions regardless of the environment.

My initial devices have explored electrical and magnetic fields for several reasons. The availability of inexpensive sensors and the obliquity of the fields were the primary requirements for the demonstration projects. In addition, these kinds of fields are commonly understood to be beyond our normal perceptual capabilities (at least, they are beyond mine – the principal test subject for this work). Concerns have been raised about the long-term effects of these fields on the human body (WHO 1998), and so in very limited way the experiments can serve as an illustration of how perceptual prosthetics might be applied in more hostile conditions and illustrate the potential of an ability to perceive conditions that we are not naturally equipped to apprehend.

Electrical and magnetic fields are pervasive in western technological culture and occur in a variety of scales relative to the body. Therefore they offer an opportunity to examine the types of movements that give rise to perceptions. For example, one prototype developed in my lab was to ascertain orientation within the earths magnetic field therefore the relevant motion was locomotive with the sensor located near the waist. In another experiment, the sensor was located in the tip of a pen in order to make use of the dexterity and training that the hand has acquired by writing, drawing and the manipulation of other tools. In this case, the target magnetic fields were those produced by a clump of rare earth magnets and required a different sensor in order to interact with fields that were smaller, dense and more spatially specific. In this case the attempt is to understand both location and intensity and required that the interface to the body had appropriate frequency and amplitude ranges.

A third device has been built for sensing electrical fields by induction. Examples of this kind of activity can be found beneath any laptop keyboard, though few have an appreciation for the richness and complexity of the activity that is resident there. To explore these fields, a glove was fabricated that contained a fingertip sensor and a tactor located above the metacarpal. On the back of the hand were the processing circuitry and power supply. In this case, the tactor initially chosen does not have, in practice, the range and dynamics necessary for the task. In addition, the sensitivity of the skin may not be capable of accepting the range of signals that could be produced in this environment by the sensor. This prototype is currently being rebuilt with a transducer of Terfenol-D, a magnetostrictive shape-change metal. The actuator will be configured to produce bone conducted audio frequencies at the base of the skull. This transducer and the capabilities of the auditory nerve more closely match the phenomena under investigation.

Preliminary device designs are promising. These, as well as the original sensory substitution devices, suggest that perceptual prosthetics are a practical possibility. A question that inevitably arises from this proposal regards the degree to which the mediated experience of extreme environments accurately represents the ‘real’ conditions of that environment. In fact, as I have argued, it is exactly the shielding from those extremes that makes this a viable rather than lethal proposal. So then, what status can we accord mediated experiences within an argument that is structured on the primacy of experience in the justification of manned exploration? Underlying this issue are assumptions about the experience of the ‘real’.

Typically, we assume that our perceptions are of reality, but our membranes are only sensitive to select portions of the available spectra. Varela, Thompson and Rotch (1991) show that the sensitivity of different species to light varies considerably. Hughes (1991) suggests that it is an exceedingly small proportion of the available spectra that we have can perceive. What we perceive directly of that which we understand by other means to be potentially available is partial and dependant upon artifacts of our biological constitution. Our perception is already ‘mediated’ by our biological makeup. What we have is our experience (Glanville 1999) and from this we construct a reality. This experience is not only of the sensory flux, but is a product also of socio-cultural activity. Perception is partially an individual construction and partly social and consensual; stabilized, in part, by a ‘normal’ distribution of biological capabilities across the population. What we perceive are the patterns that we make, however, we don’t invent them arbitrarily.

Technological mediation participates in an analogous fashion. What is available from a transducer is an uneven response of a material set-up to a some aspect of a physical condition. In order to become available as a perception, it must fit into the other patterns that we hold, to be available in the time scales in which we live and have a variable character within the scale of space that we occupy with our activity. Processing the output of a sensor, transforming and scaling, and then making it available to the body through existing biological ‘bandwidth’ are functions of the design and of the craft of the interface. There are inevitable choices that have to be made and existing materials, structures and techniques that have to be brought to bear on the problem at hand. The experience that develops out of each device will be checked against and integrated with other perceptions and other ways of knowing. In the end the verisimilitude we seek is a function of the degree to which coherent patterns can be built. The implication of this may be that we can only perceive and believe those things that cohere for us, that certain kinds of disorder may simply not be available for our apprehension. This is primarily a reflection of our humanity. Perceptual prosthetics are simply an attempt to extend those things that might be brought into the world of our experience.



Bach-y-Rita, P. (2003), “Seeing with the Brain”, International Journal of Human Computer Interaction, Vol.15, pp.285-295.

Bec, L. (2007). Extreme Environments. Mutamorphosis Conference Reference Text. Document Aavailable at: (14.08.07) .

Glanville (1970). “As is and as if: to see or not to see”. Proceedings of the Conference ‘Invençao Thinking the New Millennium’ Sao Paolo, Brazil. Document available at: (14.08.07

Hughes, H. (1999), Sensory Exotica: A World Beyond Human Experience, MIT Press, Cambridge, MA.

Lenay, C., Canu, S., Villon, P. (1997), “Technology and Perception: The contribution of sensory substitution systems”, Second International Conference on Cognitive Technology (CT ‘97), Aizu, Japan

O’Regan, K., and Noë, A. (2001), “A sensorimotor account of vision and visual consciousness”, Behavioral and Brain Sciences, Vol. 24, No. 5, pp. 939 -1031.

Varela, F., Thompson, E., Rosch, E. (1991), The Embodied Mind:  Cognitive science and human experience, MIT Press, Cambridge, MA.

World Health Organization (1998). “Electromagnetic fields and public health: extremely low frequencies (ELF) “. Document available at: (14.08.07).



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: