Spontaneous Gestures - Part 2 of 3

(Continued from Part 1 of 3)

McClave reiterated that gestures are connected to the thought expressed. She micro analyzes at one-thirtieth of a second because video goes by at thirty frames per second. She and her team, “. . . found the gesture will line up with the co-expressive part of the speech, but, it always either precedes it or comes right down on that beat where that speech is co-expressive.”

Early misconceptions about the relationship between gesture and speech were that gesture translated your speech. McClave clarified, “The gesture researchers realized that we see features and gestures that never were in speech. Therefore, it can’t be possible that it’s translating the speech because it was never there to begin with in the speech.”

McClave commented “. . . the listener picks up the gesture and no longer knows how that information got to them, through the gesture or through the speech. It only becomes totally integrated the mind. We’ve done enough experiments on this to find that people are paying attention to the gesture, incorporating into their understanding of the interaction and no longer able to tell you what came through the gesture medium what came through the speech medium.”

McClave showed us a video clip and pointed out, “Now, before this, linguists were working which is working with audio data. They would ask “Guess?” jeans? He’s talking about Guess? jeans.” However, it’s not what he’s talking about, she continued, “He’s talking about the logo across his shirt. So, now, you if you ask an interlocutor what he said, he’d say ‘yeah, they were wearing this logo across their shirts’ . . . .”

After showing us several video clips, McClave responds that linguists started to realize there were different dimensions of gestures—iconic, metaphoric, abstraction, deictic, and beats.

In a video clip, we saw a woman using an iconic gesture when she’s “holding a child.” Another clip, we saw a man describing something abstract by using his own body to show location (self-marking).

We’re shown a metaphorical gesture in another video clip. McClave notes, “So, if you know something about sign language this will be interesting. He’s going to talk about the shine in a little child’s eye. That’s not a physical things, so, he’s going to gesture for it.” She stops the video and says, “So, he’s doing something very close to ASL and there’s more about the connection with ASL and gestures later.” In the fourth video clip, the speaker refers to “the cycle of life.” After we see the speaker sweep her right hand to the right, McClave comments, “Here linguists love this. Why? First of we got one metaphor in speech. “Cycle of life,” a metaphor. We’ve got a different metaphor on the hands. The body coming out with two simultaneous metaphors. . . .”

McClave pointed out, “The deictic dimension, linguistics divide into concrete and abstract pointing. Concrete means you’re actually pointing at somebody that is right there with them or is something in the environment. But it turns out much like ASL signers, hearing people point at things that are not there at all.”

In another video clip, McClave noted that a man points to someone not there when he says, “So I’m gonna ask this one lady.” McClave interjected that, “He’s going to go over to his left. In fact the head moves first and the point goes to left. This is amazing if you know ASL. ASL has grammaticized the same thing. Which leads us to believe, to postulate, maybe ASL is based on the observation of hearing people. But nobody knew hearing people were doing this when they started to do the linguistics of ASL.”

Beats were the last dimension of gesture described. McClave asserted, “It was originally assumed that beats emphasized particular syllables. That when you wanted to emphasize a syllable you came down on a beat.” However, she discovered that the beats occurred on unstressed syllables and during silence, manifested in rhythmic patterns. Therefore, if you have to come down on the tone unit nucleus, the thought is already in the brain—the whole utterance is build up around the word.

McClave mentioned Brian Butterworth, a researcher in England. He claimed gestures happened because we are looking for the right words. She noted, “Those gestures happen but they’re very rare. . . . So, based on the evidence I've shown you, David McNeil of the University Of Chicago has put forth a theory of how we actually think. He calls it the Growth Point Theory because we can predict that the gesture will match up with the word that is semantically related to. And then there’s a predictable timing relationship. He hypothesized that probably this word and this image were a unity in the brain to begin with. And that what’s within your mind and then you go and build up your utterance around this thought, basically.”

Next we watched a video of a woman discussing a location in the O’Hare Airport. McClave described how the woman’s utterance is build up around the image in her mind using the Growth Point Theory.