The Language Problem: Jaguars & The Turing Test

“I love Jaguars!”

When I ask you to understand that sentence, I’m requiring you to take on a pretty significant undertaking, although you do it hundreds of times each day without really thinking about it.

The problem comes with the ambiguity of words.

“I” is pretty straightforward. I’m referring to me. Not much ambiguity there.

“Love” is a little more difficult, but not much. Given the grammatical structure and syntax, we can easily reduce the possible meanings of “love” (of which there are 28, according to Dictionary.com) down to a subset that have slightly different meanings, but all basically translate as, “have a strong liking for.” While there is a margin of error here, it’s minimal.

The word “Jaguars” is a different story. To parse my intended meaning, you have to do some substantial guesswork. According to Wikipedia, there are over 30 potential meanings for the word Jaguar.

For other words, they may pull double duty as both nouns and verbs (love is one example) and the simple subject-verb-object structure indicates that the right candidate would be a noun, not a verb. In this case, we could eliminate all the meanings that are verbs and stick to the nouns. But in this case, all 32 possible meanings for “jaguar” are nouns.

When I say "jaguar" do you know what I mean?

When we analyze language, we subject it to increasingly complex methods of analysis. The first is studying language structure, or grammar. We look at how words are formed (morphology), how phrases and sentences are structured (syntax) and determining the meaning of the word based on how it sounds (phonology). If there is no ambiguity with the words involved, that should be sufficient to interpret the meaning. When we learn to read, we build our skills at this first level of interpretation.

Any potential ambiguity (i.e. knowing the “Dick” in “See Dick run” is a boy named Dick and not one of the other dozen or so meanings, including a detective, an English dessert, or worse) is avoided through the use of accompanying pictures. That’s why we learn to read with picture books. It reduces the linguistic complexity of our world.

But this first line of linguistic analysis comes up short on the challenge I threw at you. We were able to parse the meaning of “I” and “love” but couldn’t determine the right meaning of the word “jaguar”. There were too many alternatives that could potentially fit in the grammatical structure of the sentence.

So now we have to move to the second level of analysis – the study of meaning. Here, we look at the context of words. First, we would look at semantics. Are there inherent clues in how the sentence itself is structures that would help us resolve ambiguity?

For example, I’ve used the plural form of Jaguar, rather than the singular form.

This is a clue that I probably don’t mean the Macintosh operating system, one of two possible comic book characters, one of three music bands, one of two potential films, the chemistry software, the Oak Ridge super computer, the Japanese wrestler or the British research rocket.  If I were referring to any of those things, I would have used the singular form of the term.

Here’s another semantic clue. If I were referring to one specific group, such as a sports team or band, I would probably use the modifier “the”, as in “I love the Jaguars!” But I didn’t. This indicates that I’m not speaking about one specific entity known as “Jaguar” or “Jaguars” but rather a group of things, animals or people known collectively as jaguars.

Because I’ve given you the sentence in written form, there’s one other semantic clue you could use to eliminate potential candidates. I’ve chosen to capitalize the word “Jaguars”, indicating that it’s a proper noun. This would also eliminate the member of the cat family from consideration. But given our rather sloppy approach to capitalization, and the fact that in spoken form, you wouldn’t have had that clue, let’s ignore it for the moment.

So semantic analysis gets us part of the way there, but we still have over a dozen potential meanings to disambiguate, including cars, animals, a game console and several different types of military hardware. Any of these are still a valid semantic fit given the structure of the sentence.

So far, a machine could do what we’ve done. The level of analysis employed to this point relies on rules and comparing alternatives to find the best match, something that machines excel at.  But it’s the next step that really separates the man from the machine.

The Human Element

Given the 20 potential meanings of jaguar, we have to go beyond the clues we find in the sentence itself and use our knowledge of the real world to narrow down the possibilities. This is where things get really interesting.

Now, when I said “I love Jaguars”, chances are you didn’t know there were over 30 potential meanings. You’ve probably never heard of the British elevator research rocket, the Icelandic funk band, the Brazilian cartoonist or the 1979 Filipino film, all known as “Jaguar”. You were probably equally oblivious to the fact that there are at least 6 different pieces of military hardware called a Jaguar.

In this way, in most instances, our limited knowledge of the world helps makes the task of word disambiguation easier. And while you’ve probably heard of the Jacksonville Jaguars NFL team, you probably didn’t know that a international rugby team in the 80’s, a second tier rugby team from Argentina, a Formula One racing team and the teams of Indiana University – Purdue and the University of Houston – Victoria are all also known at the Jaguars. In the case of word disambiguation, ignorance is often bliss.

So, in all likelihood, when I professed my love for Jaguars, you had to choose from three alternatives: the Florida NFL team, the wild animal and the British luxury car. Here, not knowing my personal inclinations, you would have gone with the most likely candidate. And your final choice would have depended on your own world view.

First of all, because I didn’t use the modifier “the”, it’s unlikely that I meant the NFL team. The way I phrased the sentence, it would be interpreted as my loving the individuals that make up the team, rather than the team itself. Without passing judgment on what attracts one human to another, the likelihood of that being my intent is probably pretty low.

Further, even if you missed that subtle semantic clue, if you follow the NFL, you know that it’s been 3 years since the Jaguars made the playoffs. You may also know that I’m from Canada. Therefore, my having a strong attachment to a mediocre team at the opposite corner of the continent seems like a stretch. It’s a possible candidate, but again, without a high level of probability.

Also, in our day-to-day lives, it’s not often that someone walks up to you and declares a strong passion for any particular wild species, especially one we never see. In certain settings (say, walking through a zoo or on an wildlife expedition in Central America) it may make sense, but not within our current frame of reference. Again, the cat is a candidate but not a highly probable one.

That leaves us with the car. Sports cars are something that middle aged men often express feelings of desire for. I fit both categories (“middle aged” and “men”). It’s not unusual for someone to say they love a particular brand of vehicle.

Therefore, given our current understanding of the world, it would make most sense to assign this meaning to the sentence to be understood. While it won’t be the right answer in all cases, it’s the one with the highest probability of success.

I’ve led you through this rather exhaustive analysis of word sense disambiguation to make a point. Understanding language is tough. And, in many cases, it goes beyond applying certain rules; it relies heavily on our experiences as human beings and making a guess about an intended meaning. That’s why it’s so difficult for a machine to do.

Yet, what it’s taken me over a 1000 words to explain, you did in a split second without really thinking about it. The application of multiple tiers of linguistic analysis was done instantly, heuristically and subconsciously. So far, no machine has been able to equal this feat.

For my next few articles, I want to take a deep dive into the problem of language. It’s something artificial intelligence experts have wrestled with for over 5 decades, and while we’ve made some headway with the first two levels of analysis (grammatical and semantic) it’s the final challenge that still lies ahead. How can machines understand language in the same way humans do?

You could consider language the “sound barrier” of artificial intelligence. Until machines can accurately interpret human language, they will be relegated to just being a tool. In humans, true intelligence has always been inextricably linked to language.

For technology to emerge and fulfill the promise laid out for it by writers such as Ray Kurzweil , Kevin Kelly and even Gene Roddenberry, we have to tackle the problem of language. It is an essential part of solving the I/O problem and finding the most human way of interfacing with a machine.

To explore the connection between AI and language, let’s roll the clock back over 60 years to the man widely regarded as the father of computer science and artificial intelligence, Alan Turing.

Meet Alan Turing

Alan Mathison Turing

In a 1950 paper titled Computing Machinery and Intelligence, Turing begins with these words, “I propose to consider the question, ‘Can machines think?”

Knowing that this was a loaded question, Turing quickly proposed an alternative: can a machine imitate a human so successfully that it could fool a human into believing it was human?

A judge would communicate with another human and a machine (both hidden) through transferring text messages (to eliminate the limitations of audio processing at the time).

If after a reasonable period of time, the judge was unable to determine which candidate was human and which was the machine, the test would be successful. This has since become known as the “Turing Test.”

Turing (who died tragically in 1952) would have turned 100 next year. He predicted that machines would be able to pass the test 30% of the time in the year 2000.

His prediction turned out to be overly optimistic. Ray Kurzweil (considered a technology optimist) now pegs the date of successfully passing the Turing Test at 2029.

Regardless of the date, understanding natural language represents one of the most significant challenges facing artificial intelligence. In the next few articles, I’ll be looking at some of the people tackling the challenge and look at how Google and other search engines currently process language.

We’ll see why the current approach to “natural language processing” still falls far short of the bar set by Turing. And we’ll explore why solving the challenge is so important to the future of computing.

By the way, I really am rather ambivalent towards Jaguars of all sorts. This introduces an entirely new challenge in communicating with machines. We don’t always say what we mean. I’ll explore that as well.