Deaf Could Hear Through Eyes, Blind Could See With Ears

If there were a universal mapping between visual data and audio data, cheap and simple devices could conceivably be made to translate from one to the other. If sight and hearing are two pathways to get stimuli into the brain; where one is limited, why not use the remaining functional pathway to deliver both streams of data?

You could give such a device to young blind or deaf children so their brain can adapt to interpret the stimuli. After all, the theory behind a cochlear implant, for instance, is that you use a mechanical microphone to grab audio signals, change them into electronic signals, and plug those new, strange signals back into the nervous system, skipping over the spot where the natural process broke down. The brain then learns to understand the new signals.

Feeding such signals into the functional senses without surgery would be much less invasive and much less expensive. For the deaf, you could have a pair of glasses with 2 microphones and a heads-up display that would visually “show” the sounds that each microphone is picking up. For the blind, you could have tiny cameras that produce a soundtrack of subtle noises corresponding to visual cues that they are picking up.



The interface would of course be clumsy at first, but with the amazing adaptability and plasticity of the young mind, the patterns could quickly become second nature.

One logical mapping of audio to visual would be to represent the spectrum of detectable frequencies (20 Hz - 20,000 Hz) as corresponding to the range of visible light (380-750 nm). You could make signal amplitudes correspond to each other, mapping a sound’s volume (dB) to a visual light’s intensity or amplitude. Both signals would be in stereo, giving a locational/spatial element, due to the presence of 2 separated inputs.

The result of mapping audio to visual stimuli would be a visualization in some obscure part of the child's eyeglasses like a picture-in-picture TV window; using clever spatial graphics, colors, and motion. The result of mapping visual to audio stimuli would be more difficult – it would have to be a series of noises that would range in various pitches and patterns, be distinct, but not be so obtrusive as to clutter one’s actual sense of hearing. They would likely carry only a limited portion of visual data for important things like large movements or changes in overall light level.

It might be good to have an easily accessible on/off switch so you could deactivate the devices if needed. To keep the devices small and less noticeable, you could have them wirelessly communicate to a mobile device and run off the computing power of an iPhone app, for instance.

CATEGORY: Health / Medicine / Technology
IDEATION: Aug 17, 2009