PUBLIC MARKS with tags "blog post" & gameai

I’ve been steadily plugging away at my list of bugs and enhancements for the upcoming release and finally tackled one area that I had been avoiding: improving the AI of some of the bugs.

In the comments thread on Greg's welcome post back last September, we had a bit of a chat about achieving real-world crowd densities on next-gen platforms. If you haven't yet seen Capcom's Dead Rising, it's worth checking out the new official trailer released last Friday - I count possibly 100+ NPCs on screen in dense urban environments, with no apparent interpenetrations and quite a bit of animation going on.

However though AI Planning has much to contribute to both these fields, particularly in producing more convincing Non-Player Characters and autonomous intelligent characters, few AI planning researchers have been involved in this work, and the technology, where applied at all, has often been used in a somewhat ad hoc way. In addition, games company use of AI planning has so far been limited - A*-based motion planning the main exception - with practitioners feeling that the technology is too computationally expensive or risky for integration into computer games.

Gamespot's Best AI of 2005

Black and White 2 AI I played Black and White 2 for many hours yesterday. The computer player and I were in a stalemate. The computer kept sending armies against me and I kept defeating them. I had built my town with walls around it, and then put archers on top of the walls. I was building up my strength while defending myself, in preparation for a big attack. I felt pretty safe. After around 40 attacks, I realized that they weren't all the same. The computer wasn't using the same attackers each time. It tried the creature, archers, swordsmen, and catapults. It tried combinations of them. Sometimes it would come through my main entrance, and sometimes it would come around the back entrance to the city. The computer player also destroyed major sections of the city using the “earthquake” power, but I recovered from these too. After a while the enemy creature figured out that he should kick my wall in. His archers and swordsmen stayed back, out of range, while the creature came up and destroyed my wall, including the archers on it. After it breached the wall, the army swarmed into my town and killed half my people. I rebuilt my wall and started to recover, but the computer's newly discovered strategy worked well. It tried several variants but kept going back to the same approach: kick down the wall, then swarm the town. This forced me to try some new strategies. Although being on the wall has advantages, it leaves the archers vulnerable when the enemy creature attacks the wall. So I moved them behind the wall. I've also learned to open my gate, wait for the enemy army to get close, then close the gate and set their army on fire. I have no good strategy for the creature knocking down my wall though, and I'm constantly losing townspeople and then rebuilding. After a long stalemate, the computer AI learned how to attack more effectively, and now I'm having trouble keeping my city safe. I'm very impressed by the AI. I'm not sure how it's programmed, but it tried out many different things and learned which ones work the best. From the game AI techniques I've learned (genetic algorithms, neural networks, fuzzy logic, state machines, etc.), the AI in Black and White 2 seems to match most closely with what I know about reinforcement learning. It's a technique that uses online learning (observing results as the game is played) instead of training (from examples constructed ahead of time), allows both exploration (trying new things in order to learn) and exploitation (taking advantage of what you've learned), and associates rewards (like whether the attack was successful) with actions (like kicking down the wall and keeping the army away from my archers). I recommend Sutton and Barto's book if you want to learn more. It's entirely possible though that the game uses something much simpler that just happens to look impressive, but my guess is that it's using reinforcement learning. — Amit — Monday, December 12, 2005 Comments: Post a Comment Links to this post:

Starting on a full time AI job "in the industry" has been an interesting experience. My new secret project is very cool, and the AI is a lot of fun, but I had been very surprised by the sheer amount of engineering details that go into a game. All the stuff that you can just ignore when doing research - well, it all comes back, with a vengeance. :)

This article introduces hidden Markov models, an inexpensive and intuitive method for modeling stochastic processes.