For those who missed it, this week on Jeopardy, former champions Ken Jennings (74 straight wins, $2+ mil) & Brad Rutter (undefeated, $3+ mil) played a 2-game exhibition tourney against Watson, a mainframe built by IBM specifically to be able to understand regular language well enough to play Jeopardy.
[spoiler]Watson won both games.
Ken placed 3rd & 2nd respectively, placing 2nd overall.
Brad placed 2nd & 3rd respectively, placing 3rd overall.[/spoiler]
Ken then wrote this essay about the experience for Slate. (He discusses the results, so those who want to wait & see a rerun or YouTube of the games should hold off on reading this.) His Game 2 Final Jeopardy response also sums things up pretty amusingly.
The computer showed some interesting behavior during the show as well:
--It cannot see or hear (it is sent the clues as txt files as Trebek reads them), so a/v clues are out of the question. This handicap also means it can (& did) repeat an opponent's wrong answer.
--While it is excellent at parsing context as intended, it struggles with figuring out the desired answer type. On at least one clue, Watson was wrong only for not giving a complete enough response.
--The IBM programmers set it only to ring in if it reached a "buzz threshold" of certainty in its results. Watson's uncertainty actually resulted in most of the human players' right answers. On Daily Doubles & Final Jeopardy, Watson is forced to give its best guess if the buzz threshold is not reached.
--Speaking of Daily Doubles & Final Jeopardy, the computer also makes some hilariously-precise wagers. Protip: The only ones-digits you ever need for betting on Jeopardy are 0, 1, & 9.
--Watson's biggest advantage, though, was physical reaction time. You could see that Ken & Brad each knew a lion's share of the answers, often faster than Watson, but a hydraulic thumb which can consistently ring in the microsecond the clue ends is kind of unfair. Jennings is known for being a quick-draw (it's how he lasted to a 75th game), it took him until Game 2 to finally start beating Watson to the buzz.
And remember, "I'm-a Luigi, number one!"
Ken Jennings Reflects on Watson Matches
-
CaptHayfever
- Supermod
- Posts: 40615
- Joined: Tue Jul 16, 2002 1:00 am
- Location: (n) - the place where I am
- Has thanked: 1220 times
- Been thanked: 803 times
- Contact:
-
Valigarmander
- Member
- Posts: 51366
- Joined: Thu Jun 01, 2006 8:22 pm
- Location: World -1
- Has thanked: 119 times
- Been thanked: 508 times
- Contact:
-
Ace Mercury
- Member
- Posts: 23140
- Joined: Mon Jun 05, 2000 1:00 am
-
CaptHayfever
- Supermod
- Posts: 40615
- Joined: Tue Jul 16, 2002 1:00 am
- Location: (n) - the place where I am
- Has thanked: 1220 times
- Been thanked: 803 times
- Contact:
Yeah, I was kinda hoping for a screwup like that in Game 2 as well. I also like how it qualifies its guesses when it was forced to answer without certainty on that Final (the question marks) & on one of the Daily Doubles ("I will take a guess").
It's obvious that Watson does not include the categories in its search parameters, too, only the clue itself.
And remember, "I'm-a Luigi, number one!"
It's obvious that Watson does not include the categories in its search parameters, too, only the clue itself.
And remember, "I'm-a Luigi, number one!"
-
Ace Mercury
- Member
- Posts: 23140
- Joined: Mon Jun 05, 2000 1:00 am
Right. It was after the fact that I read up on some links off of the Metafilter thread re: this; apparently including the category screwed up the answers for some reason.
Irregardless, I think this is pretty exciting stuff, this Watson answer generator. Even if the quality stays the same, this sort of answer generator would be an awesome digital assistant. In a few years, Watson's going to shrink down from a room full of server racks into a more portable device, and we'll have a portable navi/agent-avatar. (Now that I think about it, it doesn't have to shrink; just port all of the answers over the cloud or something.) Working together with a human to weed out stupid stuff, I think it's a great advance for personal efficiency.
Irregardless, I think this is pretty exciting stuff, this Watson answer generator. Even if the quality stays the same, this sort of answer generator would be an awesome digital assistant. In a few years, Watson's going to shrink down from a room full of server racks into a more portable device, and we'll have a portable navi/agent-avatar. (Now that I think about it, it doesn't have to shrink; just port all of the answers over the cloud or something.) Working together with a human to weed out stupid stuff, I think it's a great advance for personal efficiency.