StarCraft NeuroEvolution Unit AI

evolving the past, one generation at a time

NeuroEvolution, or better yet Real Time NeuroEvolution of Augmenting Topologies(rtNEAT) uses generic algorithms and selective breeding to both train and shape neural networks into their optimal configuration. The original C++ rtNEAT algorithm was developed at the University of Texas, at Austin by Dr. Kenneth Stanley.

My experiment in rtNEAT ported the C++ centered algorithm into a Java implementation and implemented both C++ and Java algorithms into StarCraft for performance and feasibility testing.

AIIDE 2010 Tournament maps

the battlegrounds

The 2010 AI and Interactive Digital Entertainment Conference hosted an AI StarCraft competition which pitted various StarCraft bots against one another in numerous setups. I used the Tier 1 tournament maps against the built in StarCraft computer to both train and test rtNEAT.


the Design

in the beginning

The design was simple, just like the goals, to build ever evolving, ever improving, autonomous agents. The approach I choose was a frugal approach based around the needs of the tier 1 AIIDE maps.

Two state managers were used:

  • the MapStateManager – kept the current state of the map geometry and was used for querying distance calculations between units and geometry
  • the UnitStateManager – kept knowledge about all current ally and known enemy units. Used for querying both ally and enemy distance, locations, and health.

Each ally was represented by a RtNeatUnit which contained StarCraft specific unit logic as well as a rtJavaNeat organism which determined the unit’s actions.

the Library

after the beginning

The library powering the neuroevolution algorithm was very closely modeled after the original rtNEAT implementation by Dr. Kenneth Stanley.

My implementation of the library though took a handful of shortcuts in order to speed up and short circuit implementation of recursion. The overall architecture of the neural networks, mutation, and evolution remained unchanged.

the Creation

in the wild

the Documentation

Because documentation always comes first!

Online JavaDocs

the Tech

Libraries

» BWAPI – Brood War Application Programing Interface

The Brood War Application Programming Interface (BWAPI) is a free and open source C++ framework for creating AI modules for Starcraft: Broodwar. Using BWAPI, programmers can retrieve information on players and individual units in Starcraft as well as issue a wide variety of commands to units, opening the doors for custom AIs with new micro and macro algorithms.

» BWTA – Brood War Terrain Analyzer

Broodwar Terrain Analyzer (BWTA) is a add-on for BWAPI which analyzes the current starcraft map and returns the set of expansion locations, regions, and choke points.

» BWAPI-Proxy – BWAPI Java Socket Proxy

BWAPI Java Socket Proxy exposes the BWAPI through socket communication to a running Java process to receive and send information.

» C++ rtNEAT – Real Time NeuroEvolution of Augmenting Topologies

Library implemenation of the Real-time NeuroEvolution of Augmenting Topologies method. In addition, it includes code for adaptive neurons.

» Java rtNEAT – Java Real Time NeuroEvolution of Augmenting Topologies

Java rtNEAT is a complete reimplementation of the rtNEAT algorithm into Java.

Utilities

» ChaosLauncher by MasterofChaos

Chaoslauncher is a program which serves as a GUI for a number of Starcraft plugins. It supports plugins using the BWL4 format which fully integrate with the launcher, and DLLs which are directly injected into Starcraft.

» BWAPI Injector

Chaoslauncher BWAPI dll injection plugin.

» Resolution Hack Loader by hellinsect

Resultion Hack to expand the StarCraft UI and viewing distance.

» W-MODE

Chaoslauncher plugin to allow StarCraft to run in window mode.

the Glorious Code – snippet

Just a taste

		final StarCraftBot bot = new rtNeatStarCraftBot();
		Game gameRef = null;
				
		try {
			// 1. get the initial game information
		    BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
	    	String playerData = reader.readLine();
	
	    	// 2. respond with bot options
	    	String botOptions = (allowUserControl ? "1" : "0") 
	    					  + (completeInformation ? "1" : "0")
	    					  + (logCommands ? "1" : "0")
      					      + (terrainAnalysis ? "1" : "0");
	    		    	
	    	socket.getOutputStream().write(botOptions.getBytes());
			
	    	// 3. get the starting locations and map information
	    	String locationData = reader.readLine();
	    	String mapData = reader.readLine();
	    	
	    	// TA
	    	String regionsData = "Regions:";
	    	String chokesData = "Chokes:";
	    	String basesData = "Bases:";
	    	
	    	if (terrainAnalysis) {
	    		regionsData = reader.readLine();
	    		chokesData = reader.readLine();
	    		basesData = reader.readLine();
	    	}

	    	final Game game = new Game(playerData, locationData, mapData, chokesData, basesData, regionsData);
	    	gameRef = game;
	    	boolean firstFrame = true;

	    	if (speedUp) {
	    		game.getCommandQueue().setGameSpeed(0);
	    	}
	    	// 4. game updates
	    	while (true) {
	    		
	    		String update = reader.readLine();
	    		if (update.startsWith("ended")) {
	    			break;
	    		}
	    		else if (update == null) {
	    			break;
	    		}
	    		else {	    				    			
	    			// update the game
	    			game.update(update);	    			

	    			if (firstFrame) {	    				
	    				firstFrame = false;
	    					    				
	    				// start the agent
	    				new Thread() {
	    					public void run() {
	    	    				bot.start(game);
	    					}
	    				}.start();
	    			}

	    			// 5. send commands
	    	    	socket.getOutputStream().write(game.getCommandQueue().getCommands().getBytes());
	    		}
	    	}
		}