I wrote a few weeks ago about the differences between playing face to face and online. And how one of the big factors now in online poker is how computers are used. Software helps the players who have it work out what moves their opponents are likely to do next. It’s a big advantage. But I can’t believe the next development in the poker news world. Now a computer has beaten players without the help of a human being!!
First of all, in early January, a computer program developed by researchers in Canada and the Czech Republic beat several players. But that was just the starter to the main course.
Later in the month, there was a much bigger experiment. One that everyone in the poker world was waiting for. A computer program called Libratus played 120,000 hands over 20 days against four of the best poker players around. This would be a real test. And again, the same thing happened. The human poker players were beaten.
According to the Guardian newspaper, Dong Kim, Jason Les, Jimmy Chou and Daniel McAulay spent “11 hours each day stationed at computer screens in the Rivers Casino in Pittsburgh battling a piece of software at no-limit Texas Holdem, a two-player unlimited form of poker. Libratus outmaneuvered them all, winning more than US$1.7m in chips. (Thankfully for the poker pros, they weren’t playing with real money)”
It is much more difficult for a computer to win at poker than it is at other games like chess. With chess, the computer and its human opponents have all the available information relating to the game. Everyone can see the whole board. There are no secrets. This is why computers were able to get good at chess several decades ago. I remember the games between the IBM computer Deep Blue and Gary Kasparov in the nineties. Kasparov won the first game. But Deep Blue won the second. It was the first time a computer had beaten a reigning world chess champion under tournament conditions.
However, with poker, it is much harder for the computer. There are lots of secrets. Players don’t get to see each other’s hands. So the computer doesn’t either. The computer has to cope with the fact that player A and player B will play the same cards differently. For this reason, it has taken much longer for them to learn how to play well at poker.
There are tens of thousands of different ways that a poker hand can develop – but a computer can deal with that complexity of information. However, if the computer wants to win, it has to do more. It has to learn the human characteristics of intuition. It even has to learn how to bluff. It has to learn how to second guess.
The amazing thing that this tournament in Pittsburgh shows is that computer programs CAN learn how to do that. They can learn how to play like humans – adding to their already huge advantages of data processing and speed of “thought”. As the Technology Review put it when talking about the Prague experiment, “perhaps most interestingly, the academics behind the work say their program overcame its human opponents by using an approximation approach that they compare to “gut feeling.”
Noam Brown, the Carnegie Mellon university student who built Libratus along with his professor of computer science Tuomas Sandholm, said “we didn’t tell Libratus how to play poker. We gave it the rules of poker and said ‘learn on your own’. The bot started playing randomly but over the course of playing trillions of hands was able to refine its approach and arrive at a winning strategy.”
One of the beaten poker players Jason Les said “Libratus turned out to be way better than we imagined. It’s slightly demoralizing. If you play a human and lose, you can stop, take a break. Here we have to show up to take a beating every day for 11 hours a day. It’s a real different emotional experience when you’re not used to losing that often”
I’m very interested in Mathematics. In probability. It’s one of the reasons I like poker so much. Computers have always been good at calculating that sort of thing. But I never thought they could learn how to do the other things in poker that you get face to face. Reading tells. Bluffing. Getting into your opponent’s head. Making them scared of your cards. Or more likely to make mistakes. Intuition. All that other stuff. It seems they can.
One advantage a computer also has is that it never gets tired. As long as it is plugged in. I’ll have to remember that if I ever play one!