Re: How can a (e.g.) chessprogram LEARN from playing with people?

Hi Mikael,

You asked how a chess-playing computer program can "learn" by playing against people. That's a really good question!

The details, as I am sure you guessed, get pretty messy, but in principle the answer is really simple. As the program plays, it remembers what works and doesn't work in the various situations it encounters. When it sees a situation that's similar to something it saw in the past and if what it did then worked, it does it again. If what it did last time didn't work, it does something else. The messy part is deciding what consititutes a "similar situation" and what "doing the same thing again" means. This is particularly true for a game like chess in which the number of possible moves at each point in the game is large. To decide, the program needs to look ahead to understand the moves and counter moves that are possible -- a process that quickly leads to having to consider enormous numbers of moves.

This has two consequences. First, it means (at least at the present time) that to build a really good chess playing program you're better off building in the knowledge needed to play rather than trying to design it to learn. That's howDeep Blue, IBM's chess-playing computer works. Second, it means that research in machine learning is generally done with other games where things aren't so "messy". The game of tic- tac-toe (noughts and crosses) was probably the first game which was used for this because of its obvious simplicity. For a game as simple as this, "similar situation" and "the same thing" are pretty easy define, and so it's easy to build a program that can learn what little is required to become unbeatable.

For more information on machine learning in games see, for example, Johannes Fürnkranz's bibliography on machine learning in strategic game playing or use one of the search engines. I used Google, and asked it for

+"machine learning" +games

The results ran into the thousands of hits, the first few of which, at least, seemed quite good.

Hope this helps!