Monday, February 14, 2011

Watson?

Watson plays Jeopardy! tonight.

Over at the Log, Philip Resnick looks at the project and why picking this game show is maybe the smartest move of all.

I'll be watching... will you?

Labels: ,

Post's permalink - posted at 3:26 PM  

7 Comments:

At 12:13 PM, February 15, 2011 Anonymous Kathie had this to say...

Ken Jennings just did a live online chat, the transcript of which can be found at:
http://live.washingtonpost.com/jeopardy-ken-jennings.html

Obviously he wasn't about to leak what he's contractually bound not to, but still some interesting stuff. I doubt a computer can be programmed (at least not yet) to have KJ's sense of humor.

 
At 12:51 PM, February 15, 2011 Blogger The Ridger, FCD had this to say...

Also, obviously, Watson can't hear the other contestants any more than he can hear Alex. Although sometimes I've heard a contestant ask the same question that was just ruled wrong, it doesn't happen often. Too bad both Watson and Ken had 100% confidence in the wrong answer!

 
At 1:45 PM, February 15, 2011 Anonymous Anonymous had this to say...

Why can't Watson be programmed to "hear" the other contestant's (or contestants' as the case may be) wrong answers so it can eliminate them from its corpus of possibilities? Barry? Anyone?

 
At 1:55 PM, February 15, 2011 Anonymous Kathie had this to say...

Sorry, that was my post at 1:45 PM. Simply forgot to type my name before hitting the "Publish" bar.

 
At 2:05 PM, February 15, 2011 Blogger The Ridger, FCD had this to say...

I don't think there's a "can't", but maybe voice recognition would have been too much with everything else going on.

 
At 6:32 PM, February 15, 2011 Blogger Barry Leiba had this to say...

Sorry for the long comment:

Voice reco coupled with the natural language understanding is too iffy.

First, Watson would have to transcribe what the contestant said, getting it sufficiently accurate. With these guys, that's probably not hard, because they speak clearly and don't have unusual accents... and, in any case, it could be trained to their speech, for the purpose of this contest. But in the general case, you won't get 100% recognition, though it's much better than it was, say, 15 years ago.

Then, Watson would have to interpret the transcript in the same way it interprets the J! questions. One goal could simply be to answer the question, "Does that answer match one of mine, which I should now eliminate?" It's questionable whether Ken's locution of "What is the '20s?" would match Watson's potential answer, "What is the 1920s?" Maybe yes, maybe no, but probably yes. In other cases, it might be less sure.

Then one can also imagine a more aggressive goal: that the incorrect answer might give a clue for Watson to form its own answer better, even if it doesn't eliminate an error. Ken's "he only had one... hand?", would probably have improved a humans answer from "What is leg?" to "What is, he only had one leg?"[1], whereas Watson just deafly gave the... um... lame answer that was deemed wrong.[2]

In any case, all of this adds difficulty to a situation that's already very difficult, and isn't likely to generate enough of an improvement to make much difference. So it wasn't worth the effort.[3]

————
[1] This is one of the reasons I think J!'s "form of a question" thing is a silly (and, really, sort of tired) gimmick. Some of the "questions" really have to be worded in a contrived way.

[2] It's actually not clear to me that it should have been considered wrong, outright. It's arguably right, because the gymnast's leg was, indeed, the anatomical oddity. At worst, it should have resulted in a "be more specific" response, which Watson might have been able to fix.

[3] Note that at this level, wrong answers are relatively rare; Ken and Brad correctly answer more than 90% of the clues they buzz in on. Because their thinking and Watson's analysis are also very different, it's reasonable to assume that the errors won't often coincide. And, indeed, this happened only once in 29 questions.

Also, that was a category that Watson didn't seem to fully grasp, though it's possible that by that answer ($800) it had. The category required decades, and Watson's first two choices for the $600 answer (which it didn't buzz on) were individual years, not decades.

 
At 9:34 PM, February 15, 2011 Anonymous Kathie had this to say...

Barry wrote: "[Voice recognition capability] isn't likely to generate enough of an improvement to make much difference."

I'd also wondered if it might not have been cost, er, computer memory-space effective. I suspect you're right.

 

Post a Comment

Subscribe to Post Comments [Atom]

     <-- Older Post                     ^ Home                    Newer Post -->