Xpilot DCNN Behavioral Cloning

Overview

This project focuses on using a Deep Convolutional Neural Net (DCNN) to learn expert behaviors in X-Pilot using the process of behavioral cloning. By cloning the expert behaviors of other bots, we should end up with an agent that can effectively play Xpilot against other bots and win. It should learn when it's most ideal to turn (left or right), thrust, and shoot. This DCNN is intended to be a general purpose network that can learn the behavior of any bot on any map of a standard size.

Currently, this DCNN has been able to effectively learn the behavior of a fuzzy system based expert bot and a bot controlled by a simple neural network. While the DCNN does struggle to learn some behaviors, and this does sometimes result in a high validation loss, the behavior observed in games after training generally matches the performance of the expert with some deviation.

We aim to eventually use this DCNN to combine the behaviors of multiple bots into a single agent, which will hopefully produce a more effective agent than any of the agents used to train it. Our current goals are to continue to reduce the DCNN validation loss and fine tune the network to produce more effective agents trained by just one expert.

Neural Net Architecture

Our neural network starts with the convolutional layers which is where our image data is input. The first convolutional layer has one input channel that goes into 16 feature layers. These feature layers use a kernel size of 3. The second convolutional layer also has 16 feature layers to 32 feature layers using a kernel size of 3 and a stride of 1.

We have Relu as our activation function between layers. Relu a more efficient activation function than sigmoid as it eliminates the need to calculate logarithms.

Results

The above picture shows the data collected by simpleNet playing against itself on the simple map.

When trained on fuzzyBot on the simple map the DCNN is able to attain a validation loss of ~18% after 10 epochs. The DCNN achieved a validation loss of ~28% after 10 epochs when trained on neuralBot, possibly due to illogical neuralBot behavior.

The DCNN controlled bot is able to effectively challenge both bots, and in both cases outpreform the bot it was trained on. This result was very unexpected and requires further investigaion to find the cause.

Future Works

We aim to eventually use this DCNN to combine the behaviors of multiple bots into a single agent. We predict that when trained on multiple bots, the DCNN will generalize to only include the most useful or effective behaviors from each of the individual bots. We hope that this process will produce a more effective agent than any of the agents used to train it.

Elements

i = 0;

while (!deck.isInOrder()) {
    print 'Iteration ' + i;
    deck.shuffle();
    i++;
}

print 'It took ' + i + ' iterations to sort the deck.';