fMRI

Broca’s area processes both music and language at the same time

When you read a book and listen to music, the brain doesn’t keep these two tasks nicely separated. In a new article just out, I show that there is a brain area which is busy with both tasks at the same time (Kunert et al., 2015). This brain area might tell us a lot about what music and language share.

journal.pone.0141069.g002_1

The brain area which you see highlighted in red on this picture is called Broca’s area. Since the 19th century, many people believe it to be ‘the language production part of the brain’. However, a more modern theory proposes that this area is responsible for combining elements (e.g., words) into coherent wholes (e.g., sentences), a task which needs to be solved to understand and produce language (Hagoort, 2013). In my most recent publication, I found evidence that at the same time as combining words into sentences, this area also combines tones into melodies (Kunert et al., 2015).

What did I do with my participants in the MRI scanner?

Take for example the sentence The athlete that noticed the mistresses looked out of the window. Who did the noticing? Was it the mistresses who noticed the athlete or the athlete who noticed the mistresses? In other words, how does noticed combine with the mistresses and the athlete? There is a second version of this sentence which uses the same words in a different way: The athlete that the mistresses noticed looked out of the window. If you are completely confused now, I have achieved my aim of giving you a feeling for what a complicated task language is. Combining words is generally not easy (first version of the sentence) and sometimes really hard (second version of the sentence).

Listening to music can be thought of in similar ways. You have to combine tones or chords in order to hear actual music rather than just a random collection of sounds. It turns out that this is also generally not easy and sometimes really hard. Check out the following two little melodies. The text is just the first example sentence above, translated into Dutch (the fMRI study was carried out in The Netherlands).

If these examples don’t work, see more examples on my personal website here.

Did you notice the somewhat odd tone in the middle of the second example? Some people call this a sour note. The idea is that it is more difficult to combine such a sour note with the other tones in the melody, compared to a more expected note.

So, now we have all the ingedients to compare the combination of words into a sentence (with an easy and a difficult kind of combination) and tones in a melody (with an easy and a difficult kind of combination). My participants heard over 100 examples like the ones above. The experiment was done in an fMRI scanner and we looked at the brain area highlighted in red above: Broca’s area (under your left temple).

What did I find in the brain data?

The height of the bars represents the difference in brain activity signal between the easy and difficult versions of the sentences. As you can see, the bars are generally above zero, i.e. this brain area displays more activity for more difficult sentences (not a significant main effect in this analysis actually). I show three bars because the sentences were sung in three different music versions: easy (‘in-key’), hard (‘out-of-key’), or with an unexpected loud note (‘auditory anomaly’). As you can see the easy version of the melody (left bar) or the one with the unexpected loud note (right bar) hardly lead to an activity difference between easy and difficult sentences. It is the difficult version (middle bar) which does. In other words: when this brain area is trying to make a difficult combination of tones, it suddenly has great trouble with the combination of words in a sentence.

What does it all mean?

This indicates that Broca’s area uses the same resources for music and language. If you overwhelm this area with a difficult music task, there are less resources available for the language task. In a previous blog post, I have argued that behavioural experiments have shown a similar picture (Kunert & Slevc, 2015). This experiment shows that the music-language interactions we see in people’s behaviour might stem from the activity in this brain area.

So, this fMRI study contributes a tiny piece to the puzzle of how the brain deals with the many tasks it has to deal with. Instead of keeping everything nice and separated in different corners of the head, similar tasks appear to get bundled in specialized brain areas. Broca’s area is an interesting case. It is associated with combining a structured series of elements into a coherent whole. This is done across domains like music, language, and (who knows) beyond.

[Update 13/11/2015: added link to personal website.]

— — —
Hagoort P (2013). MUC (Memory, Unification, Control) and beyond. Frontiers in psychology, 4 PMID: 23874313

Kunert R, & Slevc LR (2015). A Commentary on: “Neural overlap in processing music and speech”. Frontiers in human neuroscience, 9 PMID: 26089792

Kunert R, Willems RM, Casasanto D, Patel AD, & Hagoort P (2015). Music and Language Syntax Interact in Broca’s Area: An fMRI Study. PloS one, 10 (11) PMID: 26536026

— — —

DISCLAIMER: The views expressed in this blog post are not necessarily shared by my co-authors Roel Willems, Daniel Casasan/to, Ani Patel, and Peter Hagoort.

Advertisements

How are invisible colours possible?

N.E.R.D.

Pharell Williams: ‘I could always visualize what I was hearing… Yeah, it was always like weird colors.’

To some people I am half colour blind even though I can see everything from blue to red like most people. For them it is odd that I can only see colours when they are directly presented to me. More than that, I can only see colours for which there are proper words. These people literally see black and white symbols in colour depending on which symbol it is. And the colours they see are sometimes those which I will never see in my life because they are invisible. Incredible? A journey through the visual brain shows how this feat is possible.

The first report of invisible colour perception came from Ramachandran and Hubbard (2001), then at San Diego. They briefly mentioned a man only known as S.S. who suffers from s-cone weakness, i.e. the cells in his eyes which are sensitive to blue are impaired. Therefore, he cannot see the full colour spectrum the way most people can.
S.S. also happens to be a grapheme-colour synaesthete, i.e. he literally sees a certain colour consistently when presented with, for example, a given number. However, these number-evoked colours were special. He described them as ‘weird’ or ‘extraspectral’. He had only ever seen them in his mind’s eye, never in the outside world.
Rather than being an isolated case, regular synaesthetes can see these so called ‘Martian’ colours as well  (Ramachandran & Hubbard, 2003). Furthermore, Oliver Sacks writes about a musician he calls Michael whose synaesthesia links musical keys with colours. Sacks writes that ‘some keys seem to have a strange hue which he can hardly describe, and which he has almost never seen in the world about him’ (2007, p. 182). Pharell Williams could be yet another case.
How can we make sense of this? To understand how the human brain can give rise to Martian colour perception, a quick tour through the visual brain is necessary. When light hits the eye, it is transformed into the electrochemical information currency of the brain. The information travels from the eyes directly to the back of the brain within about 100 milliseconds (a tenth of a second). Because this area in the back of the head is so well understood to be the primary target for visual information, it is simply called the primary visual cortex or V1 for short.
visual pathway

Note how information travels from the front to the back.

After some low level information processing in V1, information is passed on to other visual areas which are specialized in certain jobs, e.g. V5 processes motion, V4 processes colour, and grapheme areas process numbers and letters.
The brain’s left hemisphere. Grapheme area in green. Colour area in red.
When looking at where these different areas lie, one gets a sense for why motion-colour synaesthesia is not found while word/number-colour synaesthesia is so common. The human brain happens to be organised in such a way that V4 (colour) lies very close to the grapheme area. V5, on the other hand, is a lot farther. Research in the last decade (reviewed in Hubbard et al., 2011) has revealed that in synaesthetes colour and grapheme areas are unusually well connected and they show activation of the colour area just after the grapheme area responds when synaesthetes view graphemes.
Martian colours are thought to be so unusual because information has not taken the usual route (eye -> V1 -> V4) but instead went to a grapheme area first and then entered V4 (eye -> V1 -> grapheme area -> V4). Similarly, musically induced Martian colours may look so weird because auditory information recoded in V4 simply isn’t of the same quality as visual information which comes from V1.
So, colour perception appears not only determined by where information is processed but also by where it originates. A good lesson to remember whenever you try to localise function X in brain area Y: information origin matters.
Some people out there see things which are so unusual that there isn’t even a proper word for these experiences. To them I am not only half-colour blind (even though I do not suffer from s-cone weakness like S.S.). I am also unimaginative – as not just my eyes but also my imagination is limited to the colours of the rainbow, a subset of the colours which can be experienced. If you believe that the rainbow is complete, you may well be as colour blind as I am.
——————————————————————————

Hubbard, E.M., Brang, D., & Ramachandran, V.S. (2011). The cross-activation theory at 10 Journal of Neuropsychology, 5, 152-177 DOI: 10.1111/j.1748-6653.2011.02014.x
Ramachandran, V.S., & Hubbard, E.M. (2001). Psychophysical investigations into the neural basis of synaesthesia Proceedings of the Royal Society B, 268, 979-983 DOI: 10.1098/rspb.2000.1576

Ramachandran, V.S., Hubbard, E.M. (2003). The phenomenology of synaesthesia. Journal of Consciousness Studies, 10, 49-57.

Sacks, O. (2007). Musicophilia. New York: Vintage.

——————————————————————————
images:
1) By kallerna (Own work) [CC-BY-SA-3.0 (www.creativecommons.org/licenses/by-sa/3.0) or GFDL (www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons
2) By Wiley (Wikimedia) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)%5D, via Wikimedia Commons
3) from Ramachandran, V.S., & Hubbard, E.M. (2001). Psychophysical investigations into the neural basis of synaesthesia. Proceedings of the Royal Society of London B, 268, p. 981.

ResearchBlogging.org