Video Game Forums

For those who missed it, this week on Jeopardy, former champions Ken Jennings (74 straight wins, $2+ mil) & Brad Rutter (undefeated, $3+ mil) played a 2-game exhibition tourney against Watson, a mainframe built by IBM specifically to be able to understand regular language well enough to play Jeopardy.
[spoiler]Watson won both games.
Ken placed 3rd & 2nd respectively, placing 2nd overall.
Brad placed 2nd & 3rd respectively, placing 3rd overall.[/spoiler]

Ken then wrote this essay about the experience for Slate. (He discusses the results, so those who want to wait & see a rerun or YouTube of the games should hold off on reading this.) His Game 2 Final Jeopardy response also sums things up pretty amusingly.

The computer showed some interesting behavior during the show as well:
--It cannot see or hear (it is sent the clues as txt files as Trebek reads them), so a/v clues are out of the question. This handicap also means it can (& did) repeat an opponent's wrong answer.
--While it is excellent at parsing context as intended, it struggles with figuring out the desired answer type. On at least one clue, Watson was wrong only for not giving a complete enough response.
--The IBM programmers set it only to ring in if it reached a "buzz threshold" of certainty in its results. Watson's uncertainty actually resulted in most of the human players' right answers. On Daily Doubles & Final Jeopardy, Watson is forced to give its best guess if the buzz threshold is not reached.
--Speaking of Daily Doubles & Final Jeopardy, the computer also makes some hilariously-precise wagers. Protip: The only ones-digits you ever need for betting on Jeopardy are 0, 1, & 9.
--Watson's biggest advantage, though, was physical reaction time. You could see that Ken & Brad each knew a lion's share of the answers, often faster than Watson, but a hydraulic thumb which can consistently ring in the microsecond the clue ends is kind of unfair. Jennings is known for being a quick-draw (it's how he lasted to a 75th game), it took him until Game 2 to finally start beating Watson to the buzz.

And remember, "I'm-a Luigi, number one!"

[quote="Ken]Watson has lots in common with a top-ranked human Jeopardy! player: It's very smart"]

Funny guy.

How long before IBM changes its name to Skynet?

I loved Trebek's reponse when Watson repeated a wrong answer from Ken; it was a smarmy "No, Ken said that :rolleyes: ".

Yeah, Watson's wager amounts are hilarious. Also hilarious: guessing final Jeopardy catagory "U.S. Cities" with "What is Toronto?????"

Yeah, I was kinda hoping for a screwup like that in Game 2 as well. I also like how it qualifies its guesses when it was forced to answer without certainty on that Final (the question marks) & on one of the Daily Doubles ("I will take a guess").
It's obvious that Watson does not include the categories in its search parameters, too, only the clue itself.

And remember, "I'm-a Luigi, number one!"

Right. It was after the fact that I read up on some links off of the Metafilter thread re: this; apparently including the category screwed up the answers for some reason.

Irregardless, I think this is pretty exciting stuff, this Watson answer generator. Even if the quality stays the same, this sort of answer generator would be an awesome digital assistant. In a few years, Watson's going to shrink down from a room full of server racks into a more portable device, and we'll have a portable navi/agent-avatar. (Now that I think about it, it doesn't have to shrink; just port all of the answers over the cloud or something.) Working together with a human to weed out stupid stuff, I think it's a great advance for personal efficiency.