Abstract
This website presents a project that involves both computer vision and artificial intelligence to give a computer the ability to emulate a human playing a computer game. The computer vision component deals with recognizing objects and the area of play in a game, taking input in the form of images displayed on a monitor. The artificial intelligence (AI) component reacts to events based on the output of the computer vision component. The project has two phases with the goal of achieving at least Phase I and, with time permitting, implementing Phase II. In either case, the game used is restricted to Dr. Mario [1], a legacy Nintendo game that shares similarities to the game Tetris. Previous work has resulted in a similar system for the game Tetris [2]. Provided below is a high-level outline of the steps that were taken to implement Phase I and the planned steps for Phase II.
- The game used was Dr. Mario
- Acquire image and detect the playing field – this is the area of interest displayed on the screen
- Find the playing field
- Determine its dimensions
- Detect objects currently in the playing field
- Find the objects that are randomly placed in the field of interest at the start of the game
- Determine type of object by color and shape
- Detect changes between frames in playing field
- Determine the location of the falling object with respect to the other already placed (static) objects
- Determine which pieces have been eliminated
- React to the changes in the playing field
- Determine a logical position to place the falling object
- Display the button that needs to be pushed by the user
Phase II
- Implement Phase I using images obtained by a “web-cam” from a screen displaying Dr. Mario
- Develop an interface from the computer to the game system so that the AI can play independently of any human input