Thought Construction in the Human Mind
If we know how the mind constructs thoughts, couldn't we read minds?

As you read this sentence, your eyes follow the words across the page and you may be aware of
a voice in your mind silently muttering along. Most of us do this when we talk, listen, or write
anything like letters or emails.  As I type this sentence, the same thing is happening in my
mind. A private internal narrative is shaping the words just before I type them.

What if we could tap into a person's inner voice? Thinking of words does, after all, create
characteristic electrical signals in our brains, and decoding them could make it possible to
piece together our thoughts. On the one hand, the ability to read minds could be used to aid
people with certain kinds of brain damage. On the other hand, it would also carry serious
implications for the future of privacy. The Transportation Security Agency (TSA) wants to
develop means for reading the minds of anyone getting on an airplane since they realize that
such a device would be getting at the heart of the matter. It's time to start thinking about these
things, because the technology to access those signals deep within the brain is fast becoming
a reality. Welcome to the brave new world of mind-reading.

Understand how the brain turns thoughts into words and you can make
machines that read minds

The Electroencephalogram (EEG) evolved until by the mid-1990s, our ability to translate the
brain's activity into readable signals had advanced so far that people could move computer
cursors using only the electrical fields created by their thoughts. The electrical impulses such
innovations tap into are produced in a part of the brain called the motor cortex, which is
responsible for muscle movement. To move a cursor on a screen, you do not think "move left"
in natural language. Instead, you imagine a specific motion like hitting a ball with a tennis
racket. Training the machine to recognise which electrical signals correspond to your
imagined movements, however, is time-consuming and difficult.

And while this method works well for directing objects on a screen, its drawbacks become
apparent when you try using it to communicate. At best, you can use the cursor to select
letters displayed on an on-screen keyboard. Even a practised mind would be lucky to write 15
words per minute with that approach. Speaking, we can manage 150.

Matching the speed at which we can think and talk would lead to devices that could instantly
translate the electrical signals of your inner voice into sound produced by a speech
synthesiser. Such an application would be a godsend for severely paralysed people, or those
who are awake and fully conscious but are unable to communicate--so-called "locked-in
syndrome"--who could once again engage in effortless conversations.

A new approach

This effortlessness, however, requires a significantly different approach. You need to focus
only on the signals coming from the brain areas that govern speech. In 2008, neuroscientist
Philip Kennedy at Neural Signals in Duluth, Georgia, tried to do just that by closing in on the
small, face-specific part of the motor cortex, which controls the mouth, larynx and tongue.
Here, it is possible to select only the muscle signals that are created when a person speaks.

For example, saying the word "bee" requires the mouth to execute very specific movements.
First the lips must be pursed to form the initial "buh" sound, then the lips must stretch and the
tongue press against the top row of molars to express the sound "ee". All of these movements
originate in the facial motor cortex, and so by tapping the signals before they mobilise their
target muscles, you can work out what the person intends to say.

Kennedy and Frank Guenther, a neuroscientist at Boston University, read these signals in a
person with locked-in syndrome by implanting penetrating electrodes in his facial motor
cortex. It worked: they were able to identify three vowel sounds Movie Camera , and feed
these signals to a speech synthesiser that broadcast the sounds in real time. The work was a
first step in deriving the consonants and vowels that make up whole words. Kennedy thinks
that by using this method he and Guenther could have a sample of 100 or more useful
words for this patient within five years.

The method is not ideal. For one thing, it is specific to one patient. Perforating a person's brain
with an electrode is also not something to be undertaken lightly. And finally, while the move
from using general motor signals to using language-specific signals is a significant step, it is
not mind reading.

Real mind reading requires some way to intercept the signals before they hit the motor cortex.
That means researchers have to search other parts of the brain for the source of language.
Fortunately, they have a good idea where to look.

The translation of thoughts to language in the brain is an incredibly complex and largely
mysterious process, but this much is known: before they end up in the motor cortex, thoughts
destined to become spoken words pass through two "staging areas" associated with the
perception and expression of speech.

The first is called Wernicke's area, which deals with semantics--in this case ideas based more
in pure meaning than in linguistics, which can include images, smells, or emotional
memories. Damage to Wernicke's area can result in the loss of semantic associations: words
can't make sense when they are decoupled from their meaning. Suffer a stroke in that region,
for example, and you will have trouble understanding not just what others are telling you, but
what you yourself are thinking. Trying to speak results in garbled, nonsensical juxtapositions
that neuroscientists call "word salad".

The second is called Broca's area, agreed to be the brain's speech-processing centre. Here
semantics are translated into phonetics and, ultimately, word components. From here the
assembled sentences take a quick trip to the motor cortex, which activates the muscles that
will turn the desired words into speech. Injure Broca's area, and though you might know what
you want to say, you just can't send those impulses, effectively rendering you mute.

When you listen to your inner voice, two things are happening. You "hear" yourself producing
language in Wernicke's area as you construct it in Broca's area.

Outing your inner voice

The key to mind reading seems to lie in these two areas, but they had remained largely
unexplored for good reason. Neuroscience was limited by the two main technologies that
could read electrical signals in real time. Penetrating electrodes pick up with high
fidelity the signals emitted by the few neurons they touch, but they inevitably carry risks.

Traditionally, the only alternative has been a surface EEG taken from electrodes sitting on top
of the skull. Unfortunately, the bone blocks and jumbles the brain's electrical signals, so while
inexact signals can be used to deduce broad muscle movements, finding meaningful
information in tiny areas like Broca's and Wernicke's requires much greater precision.

By 2004 neuroscientists had turned to a technique called electrocorticography (ECoG), which
involves opening the skull and placing arrays of electrodes onto the top of the brain, just
beneath the skull but without actually penetrating the brain (Journal of Neural Engineering,
vol 1, p 63). ECoG has been used for several decades to locate the source of seizures in people
with epilepsy.

Armed with ECoG, several researchers set about looking for the inner voice. Last year, Bradley
Greger at the University of Utah in Salt Lake City successfully used ECoG to detect the
electrical signatures of whole words--finer detail than anyone had ever obtained. By placing
arrays of electrodes over both the facial motor cortex and Wernicke's area, Greger and his
colleagues detected the words "yes", "no", "hot, "cold", "thirsty", "hungry", "hello", "goodbye",
"more" and "less" as they were being produced in the brain of an awake, aware person with
epilepsy.

Like Kennedy's group, Greger says, "we were going after a constant set of words", specifically
chosen to be useful to locked-in people. But unlike Kennedy, Greger was looking beyond the
motor cortex. Though most of the signals still came from the facial motor cortex, this work
nevertheless marked the first ever excursion into the brain's language centres.

Promising as it is, this approach does not scale up well. Like Kennedy's word-mapping
technique, it requires a new signal to be learned for each new word you add to the lexicon.
English contains a quarter of a million distinct words. And though this was the first instance
of monitoring Wernicke's area, it still relied largely on the facial motor cortex.

There might be another way. "You could go after more fundamental building blocks of
language," says Greger. These building blocks are called phonemes and the English language
has about 40 of them--the "kuh" sound in "school", for example, the "ah" in "blah", the "sh"
in "shy". Every English word contains some subset of these sound components. Decode the
brain signals that correspond to the phonemes, and you would have a kind of Rosetta Stone to
unlock any conceivable word at the moment you think it.

Where would you find these phonemes? They would be much more likely to exist in Broca's
area or Wernicke's area, the precursor regions to verbal motor signals.

In fact, tantalising clues from functional MRI scans had hinted at the presence of phonemes
in these areas. So last year Eric Leuthardt at Washington University School of Medicine in St
Louis, Missouri, and his colleague Gerwin Schalk positioned ECoG patches over the language
regions of four fully conscious people. Sure enough, it was possible to detect the phonemes
"oo", "ah", "eh", and "ee". The neural signatures of those sounds in the motor cortex looked the
same whether the participants imagined they were speaking them or actually spoke them
aloud. However, phonemes that were strictly imagined--sounds that the thinker was not
intending to speak aloud--produced radically different signatures.

Leuthardt had been on the hunt for phonemes, but he had stumbled across something much
better. Spoken phonemes activated both the language areas and the motor cortex. But
imagined speech--that inner voice--boosted the activity of the neurons in the Wernicke's
area. "This imagined speech, or inner voice, was especially easy to pick up in the classical
language areas," he says.  "Part of the way we process the world is through that internal
dialogue," says Leuthardt. And he was able to listen in on it. He had effectively read their
minds. "I would call it brain reading," he says. "It is literally taking your thoughts and
translating them to the outside world." Leuthardt's work has impressed other researchers.
"This is the first time anyone has been able to get these kinds of signals out of the language
areas," says Greger. "It's really impressive."

The real-time speed at which Leuthardt was able to detect the specific phonetic signals
implies that it could be done as we think the words. We could then read signals off an ECoG at
the instant a person thought them. "It should be possible to produce real-time speech,"
Kennedy says.

To arrive at whole words, Leuthardt's next step is to expand his library of sounds. He initially
thought pursuing vowel phonemes would be far easier than consonant type phonemes
because the mouth movements are so different, whereas transitioning between "k" and
"t" barely causes the mouth to move. However, that is not necessarily a problem. It's too early to
discuss the research findings, but progress is happening faster than he had anticipated,
and Leuthardt is very happy with the results. "We're burning through [phonemes] fast," he says.

For now, this research is primarily aimed at improving the lives of people with locked-in
syndrome, but the ability to explore the brain's language centres could revolutionise other
fields. The consequences of these findings could ripple out to more general audiences, who
might like to use extreme hands-free mobile communication technologies that can be
manipulated by inner voice alone; consider a device that could someday give you your own
private telepathic link. For linguists, it could provide previously unobtainable insights into
the neural origins and structures of language. For example, linguists currently have only a
hazy understanding of the neurobiological processes that create language from thought.

Leuthardt's next project is a collaboration with international researchers to find out how the
production of phonemes translates across different languages. If linguistic thoughts activate
the same groups of neurons in people across cultures and languages, that might imply that
humans are in some way hard-wired to communicate using these types of sounds.

Leuthardt is not, however, content to stop at the inner voice. He is also looking further, to see
whether semantic understanding can be mapped between vastly different cultures: which
groups of neurons fire, for example, at the idea of "dog" as thought by an American and a
Chinese person? "We want to see how far we can reach," Leuthardt says. "Let's see what
semantic representations look like." If it were possible to generalise these semantic images
and use them for communication instead of language, universal translators might be within
reach. Indeed, knowing what someone is thinking without needing words at all would be
functionally indistinguishable from telepathy.

Is the world ready for mind reading? Or is it the stuff of Orwellian nightmares--a technology
that makes it possible to read your deepest, darkest secrets? Delve far enough into the brain's
idea-making processes, after all, and it should be possible to probe thoughts that were never
intended for sharing. Where Kennedy's method relied on speaking or the intent to speak,
Leuthardt's technique can pick up those thoughts without the participant ever intending to
speak them.

For now, this difference is academic. This version of mind-reading requires brain surgery. But
if less invasive techniques could harness the same information, all bets would be off. If that
were to happen, it would be important to ensure that there was a mechanism to stop
information being extracted without the subject's cooperation.

In fact, such a mechanism already exists. If we always said every word we were thinking, the
consequences would be disastrous. Instead, we separate public from private with a kind of
self-censoring mechanism, though the point at which this mental brake comes into play
during speech generation is unclear, says Leuthardt. It's an important distinction, because if
the brake exists only in the motor cortex, information accessed at the language centres would
be unfiltered.

What do you think? Are we ready to breach the last bastion of privacy--our own skulls--to
reveal the things we have so far kept safely locked away in the darker corners of our heads?
No, don't say anything. I've already read your mind.

Let the mind-reading begin
http://www.newscientist.com/article/mg21028142.800-let-the-mindreading-begin.html

JEAN-DOMINIQUE BAUBY famously wrote his memoir The Diving Bell and the Butterfly with
painstaking blinks of his left eye--the only part of his body over which he had any control. We
report on a technique that could allow other locked-in people to communicate more naturally,
by directly translating their thoughts into words (see "Mind readers: Eavesdropping on your
inner voice").

This stunning research is mainly aimed at liberating minds trapped in broken bodies, but it
could also blow the lid off one of the oldest questions in linguistics and philosophy: what is the
relationship between language and thought? Does our native language influence how we
think, or is the essence of thought the same for us all? The debate has run for decades and,
because it was largely untestable, it has been a matter on which smart people must agree to
disagree.

The new work may finally deliver an answer by offering access to the neurological essence of
thought before it is converted into speech. It may even be a step on the road to a universal
translation device.

Concerns will inevitably be raised about "mind-reading" and privacy. But in light of the
benefits, the risks are worth taking. Let the mind-reading begin.

Next