Using fMRI to predict word representation in the brain

Just et al have an interesting paper (here) using brain images to predict what noun participants were looking at (for 3 s) in their visual field. Importantly, they did not see an actual picture of an object based on the word, although they were instructed to think about what qualities the word connotes. Their model takes into account four factors of a word to predict its brain activation: manipulation, shelter, eating, and word length. Here’s a tantalizing picture of one participant’s expected and actual brain activations upon seeing the words “apartment” and “carrot.”

coronal slice in the parahippocampal area (dark blue ellipse) and precuneus area (light blue ellipse); doi:10.1371/journal.pone.0008622

The real test of any true model is in prediction. For data within one participant, in answering the question “What will the activation patterns be for these two new words, given the relation between word properties and activation patterns for the other 58 words?” the model had a mean accuracy of 0.801. Still within one participant, in answering the question “Which of the 60 words produced this activation pattern, given information from an independent training set?” the model had a mean accuracy of 0.724. These accuracies are far, far above chance.

Even across participants, the model was accurate. In generating predictions concerning two previously unseen words for a previously unseen participant (from training data of the 10 other participants and 58 other words), the model had a mean accuracy of 0.762. Possibly brain imaging needs a Turing test to decide exactly what would be required to say that researchers can “read minds” in an fMRI machine? I would say, > 98% accuracy for what word subjects focus on when they decide to focus on a word, out of all possible dictionary words, would be pretty close to mind reading to me. Or maybe they don’t have to actually guess the word entirely correct but just get its meaning down in terms of various components? All I know is, it will probably be easy to tell when people have the munchies, because there will be a lot of activation in the “eating” factor.

Reference

Just MA, Cherkassky VL, Aryal S, Mitchell TM (2010) A Neurosemantic Theory of Concrete Noun Representation Based on the Underlying Brain Codes. PLoS ONE 5(1): e8622. doi:10.1371/journal.pone.0008622