The next video, that shows about 166 generations of the evolving neural networks, clarifies the situation enough so I can explain myself later:

I’m interested in programming above all because of artificial intelligence and videogames. Although I consider myself a writer first, another one of my dreams, shared with many programmers of modern videogames, consists on making a game with the best of Dwarf Fortress, but with the more immediate and exploratory aspects of games like Cataclysm: Dark Days Ahead. Rimworld’s ambience comes somewhat close to what I would want, but I’d prefer a complexity closer to that of Dwarf Fortress, with most of its elements depending on procedural generation, and supported with an artificial intelligence based on neural networks that would offer constants surprises.

To prove to myself that I could program the visual aspect of a similar game and set the basis for developing the intelligence of its actors, I intended to develop the prototype shown in the video. For now I’ve failed in generating the behaviors that I wanted for the involved neural networks, but having reached this point allows me to progress quickly.

For those who don’t know it, and according to the story as I remember it, neural networks were considered the Holy Grail of artificial intelligence in the eighties and early nineties, but they crashed against an unsolvable problem: no mathematical model could determine which was the best architecture to use for a network to solve a particular problem. They depended on trial and error, and eventually neural networks ended up relegated to obscurity for the most theory-minded and a minority of dedicated programmers.

But in 2002, Kenneth Stanley, from Texas university in Austin, wrote the following paper:

Evolving Neural Networks through Augmenting Topologies

The paper, and those that followed it, revealed the way to solve the main issue with neural networks: instead of designing its architecture, it should evolve through a genetic algorithm. A significant part of the revolution in artificial intelligence we are living in has its origin in papers like this one and others from that era.

The prototype I intended to build had to implement the the following elements:

• The visual aspect of games like Dwarf Fortress (with tilesets) and Cataclysm: Dark Days Ahead.
• A reusable architecture that would allow adding new features and programming other experiments easily
• It would implement the neuroevolution using some library based on NEAT
• The neural networks should evolve a behavior close to what I intended

Visual representation of a neural network:

A neuroevolution tends to begin with only the input and output layers set. The inputs represent the sensorial information that a neural network would process, and the outputs the answers that the internal architecture generates through the interaction of all the nodes.

The success of this method depends mainly on the following factors: the inputs must feed the network with the relevant information that could end up producing the intended behavior, and the inputs should be normalized (should be scaled to a range of 0.0 to 1.0). For my experiment I settled on the following inputs:

1. The normalized value of each turtle’s health
2. A value of 1.0 if the animal detects a fruit close enough in the northwest, but 0.0 otherwise.
3. Same but in the north
4. Same but in the northeast
5. Same but in the east
6. Same but in the southeast
7. Same but in the south
8. Same but in the southwest
9. Same but in the west

I decided that each actor would act depending on which output had produced the highest value, and according to its index, the actor would walk a tile over to one of the cardinal directions or it would stay in place.

I chose those inputs because I considered that an actor should learn to link his health deteriorating quickly to the need to search for food, and the actor should detect the fruits instead of stumbling in the dark.

Apart from the architecture of the network, the other key is the function that determines each neural network’s fitness. The fitness is a mathematical measure of how close the network has gotten to the goal. Usually it consists in reaching a high number. In my case I settled for the following function:

((Health ^ 2) + (AmountOfFruitsEaten * 50) + (TurnsSpentNearFood * 2)

I wanted to reward the actors that kept their health as high as possible, and as secundary measures I wanted to suggest that they should look for food and keep close. I’m terrible at math and I’m doing this alone, so suggestions are welcome.

For now the experiment hasn’t produced the behaviors I intended. I’ll leave it some night so it can reach a thousand or thousands of generations, but at least I’m happy that it’s built upon a platform that could allow me to move towards programming something close to an interesting videogame or simulation.

UPDATE: I’ve changed some aspects of the experiments and gotten results close enough to what I intended. I’ll write another post showing them.