How to Program an A.I. to Beat the World's Highest Flappy Bird Score
While I might not be good at playing ‘Flappy Bird’, I am good at programming. Which technically means…I am really good at Flappy Bird.
So, I built a Flappy Bird game along with an A.I for it that is capable of learning and playing it in a day. Here’s how I did it.
If you want to check out the source code, here's a link to my GitHub.
My Approach to This Project
To create this project, I used Neuroevolution of Augmenting Topologies (NEAT) and Python to program a neural network capable of teaching itself to understand and survive simulated scenarios.
I programmed classes for Flappy Bird in Python, then implemented some physics and death mechanics along with pixel precision collision accuracy to it.
I implemented a Neural Network using NEAT and bred birds every generation until a certain fitness threshold was obtained and the Neural Network was capable of figuring out a solution to the simulated scenario.
I used the following technologies and tools to complete this project:
Python 3.8.5 and MacOS Terminal as its interpreter
Vim and Jupyter Notebook as primary IDE's (Personal Preference)
Python NEAT Library ()
Other common libraries such as pygame, time, random, os
Prerequisites:
Basic Python knowledge about classes, functions & objects
How the game Flappy Bird works (to comprehend the Physics and the Death Mechanics implemented in this project).
What Is NEAT and How Can We Implement it Here?
NEAT stands for NeuroEvolution of Augmenting Topologies. In basic terms, it is essentially an attempt to copy evolution in nature.
What we'll do here is generate a bunch of random players and test them. The players that do the best will be sent for breeding and the others will be mutated.
This means that the next generation will be filled with the best players. And the process repeats until we have a network capable of doing what we want.
How Do Neural Networks Work?
Before explaining NEAT’s implementation in our project, I first want to start off by explaining how Neural Networks work.
Essentially Neural Networks come in layers, with the first layer being the input layer.
Input layers are the information passed to our network, like what the network actually knows and sees. For our project, we could tell the bird about the distance between the bird and the top/bottom pipes as the input layer.
The last layer is the output layer, which will tell the A.I. what it should do. In our case, it will tell the bird whether it should jump or not.
A Deeper Explanation of Neural Networks – a Mathematical Approach
While working with neural networks, we start off by passing some values into the input layer neurons which will then pass them to the output neuron using Connections.
All Connections will have something called Weights, which is a number representing how strong or weak the connection is. We will tweak these numbers to make our A.I. perform better.
After this, we will calculate the Weighted Sum. This means that we are going to take each weight and multiply it by its corresponding input value and then add them.
Feed-Forward Neural Network
Then to the weighted sum, we add two things:
(i) Bias: A number that lets us tweak our Neural Network to allow us to move our network a little bit up and down and shift it to the best possible position.
(ii) Activation Function:This function will help us get the value for our neuron between a range of two numbers. This is very useful, as in this case we can tell the A.I to jump or not jump depending upon the value returned by the activation function (bird jumps if output > 0 in our case).
For this project, we will use TanH as our activation function.
TanH function compressing the output values between -1 and 1
So, you might be wondering – how do we get the correct weights and biases for our bird here?
Well, we actually don't. We tell the computer to do it for us. This is where NEAT comes into play!
Here we start off by creating a population of birds that is completely random. Every bird of the population will have a different neural network controlling it.
These neural networks will all start with random weights and biases. Then, we put all of them on our level and see how well they perform. We relatively tweak their fitness to encourage them. Fitnessis essentially how “fit” or how “good” the solution is with respect to the problem in consideration, and is different depending on the task.
So when do we increase the bird's Fitness value?
In our case, we will increment the fitness of the birds that have successfully passed a pipe and reduce the fitness of those birds that have failed.
To encourage the bird to stay alive and move, we will also increase its fitness if it has moved forward every frame without dying.
Finally, at the end of the simulation, the birds that have performed the best will be bred to form a new population (generation +=1). This generation of birds will perform better and the cycle continues until we obtain our desired fitness level.
There's also an ‘n’ number of Hidden Layersbetween the input and the output layer. Hidden layers capture more and more complexity with every layer by discovering relationships between features in the input.
The NEAT Cycle
To summarize, we are using a ‘feed-forward’ neural network, where the input is multiplied by the weights, summed, and passed through non-linear equations of choice, that is activation functions (eg. Sigmoid, ReLU, TanH).
Hidden layers apply a non-linearity to the neural network’s input, and the more hidden layers you stack together, the more complex functions you will be able to model.
How to Program Flappy Bird and Apply NEAT
I am going to assume that you have prior Python knowledge.
Let’s start by importing the necessary libraries and assets. Since all the image assets found online were a bit small, we will multiply their size using pygame.transform.scale2x().
Now that all our assets are loaded, we will start working on the class Bird():In this class, we will define the following methods:
rip_animation(self): This is the method that will be called if a bird hits a pipe. The bird will go vertically downwards until it hits the ground.
move(self): To make things clear, the bird actually doesn't have to move forward, but rather just up and down because the pipes and the ground are all that's moving. Here we will create a physics mechanism where we’ll adjust the birds' displacement from the y axis depending upon the time passed by calculating the number of ticks.
jump(self):We will add -ve velocity to the bird here because the top left corner of the pygame window has coordinates (0,0) and the bottom right will have coordinates (500,800). Thus to make the bird go upwards, we will have to reduce its y-coordinate.
draw(self, win):In this method, we will tilt the bird according to where it's going. One more thing we need to do is tell the bird not to play the flapping images if it's going down since that would look kinda dumb.
get_mask(self):Here we will use an inbuilt pygame function to mask every pixel of the bird since we desire pixel-perfect collisions.
You also might be wondering how we can implement a bird’s velocity. Well, the solution is to set velocity to some value and, using it, implement a formula to calculate a displacement ‘d’.
Then we run the game at 60fps, and we call the main( ) 60 times a second. This will also invoke the method containing the displacement and so move the bird a certain distance ‘d’ every frame (1/60th of a second). This will make it look very smooth while moving with some velocity.
"""
Code for the Bird Class
"""
classBird():
animation_imgs=all_bird_imgs
# while the bird moves up and down
maximum_rotation=24
# velocity with which we rotate the bird
rotation_velocity=18
# flap animation duration
flap_animation=8
def__init__(self, x, y, rip):
# x, y are the starting coordinates, rip (rest in peace) is the boolean value that checks if the bird is alive or not
self.x=x
self.y=y
self.rip=rip
self.tilt=0
self.ticks=0
self.vel=0
self.height=self.y
self.img_count=0
self.img=self.animation_imgs[0]
defrip_animation(self):
# positive velocity will make it go in down as the y coordinate of the pygame window increases as we go down
self.vel=10
self.ticks=0
# if bird is rip (by hitting a pipe) then it will turn the bird red and move it to the ground where we will remove it from the list of birds
self.height=window_height
defmove(self):
self.ticks=self.ticks+1
# d stands for displacement
d=self.vel* (self.ticks) +1.5*self.ticks**2
ifd>=14:
d=14
ifd<0:
d-=2
self.y=self.y+d
ifd<0orself.y<self.height+50:
ifself.tilt<self.maximum_rotation:
self.tilt=self.maximum_rotation
else:
ifself.tilt>-90:
self.tilt-=self.rotation_velocity
defjump(self):
# since top left corner of pygame window has coordinates (0,0), so to go upwards we need negative velocity
self.vel=-10
self.ticks=0
self.height=self.y
defdraw(self, win):
# img_count will represent how many times have we already shown image
self.img_count=self.img_count+1
# condition to check if the bird is alive
ifself.rip==False:
# checking what image of the bird we should show based on the current image count
ifself.img_count<=self.flap_animation:
self.img=self.animation_imgs[0]
elifself.img_count<=self.flap_animation*2:
self.img=self.animation_imgs[1]
elifself.img_count<=self.flap_animation*3:
self.img=self.animation_imgs[2]
elifself.img_count<=self.flap_animation*4:
self.img=self.animation_imgs[1]
elifself.img_count==self.flap_animation*4+1:
self.img=self.animation_imgs[0]
self.img_count=0
# this will prevent flapping of the birds wings while going down
The next class is Pipe():. In this class, we will define the following methods:
set_height(self): This method randomly sets the heights of the top and bottom pipes and makes sure that the gap between them stays constant.
draw(self, win):Draws the top and bottom pipes on the window
collide(self, bird, win):This method will get the bird’s mask and check if it overlaps with the top or bottom pipe’s mask. If yes, then it will return True (which will be used later), otherwise, return False.
"""
Code for the Pipe Class
"""
classPipe:
gap=220
# velocity of pipes, since the pipes move and the bird does not
vel=4
def__init__(self, x):
# x because the pipes are going to be random
self.x=x
self.height=0
# creating varibles to keep track of where the top and bottom of the pipe are going to be drawn
The next class is Base(): In this class, we will make the base appear to be moving infinitely. But in theory, we are actually looping two pictures of the base one behind another. Check out the diagram below that explains this.
Looping the base to make it look infinitely long
The methods we will define in this class are:
move(self): This method will move the images by some distance to the left per frame. Then, when the first image is completely off the screen, it quickly goes behind the second image, and this loops until termination.
draw(self, win):Draws both the base images on the window.
The next step is to actually draw everything on the window, like all the pipes, the score, and other stats.
The important thing to do here is to make sure that you are writing this outside the classes, that is starting with zero indentation. The code is pretty self-explanatory.
"""
Code to Render Pipes, Birds, Stats, etc. on the Window
Now it's time to program our main(). It will also behave as the fitness function for our project. In this, we will do the following:
Setup a FeedForward Neural Network for the genomes using the config file located in assets (imported after main).
Then we will place the initial pipes and the base and set a clock to tick 60 times a second.
Our next goal is to make sure the bird looks at the pipes that are in front of it and not the pipes that it has already passed.
Then, we will instruct the bird to jump if the output returned by the neural network is > 0. Now, we will increment the fitness of the bird if it either goes in between the pipes or if the bird is alive for a given frame (this will encourage it to stay alive and flap its wings)
If a bird does hit a pipe, we will reduce its fitness (so it doesn't breed to form the next generation) and set the bird’s ‘rip’ attribute to True, which will turn the bird red and trigger the rip_animation to make it fall to the ground.
Whenever a bird hits the ground or tries to trick the system by going above the screen to cross the pipes, we will remove it from the birds’ list by using the pop() function (Also make sure to pop the network and genomes associated with that bird before popping the actual bird).
"""
Code for main()
This will also behave as a fitness function for the program
"""
# the main() will also act as a fitness function for the program
defmain(genomes, config):
globalgenerations
generations+=1
nets= []
ge= []
birds= []
# setting neural network for genome, using _, g as 'genomes' is a tuple that has the genome id as well as the genome object
for_, gingenomes:
net=neat.nn.FeedForwardNetwork.create(g, config)
nets.append(net)
birds.append(Bird(210, 320, rip=False))
g.fitness=0
ge.append(g)
base=Base(730)
# first pipe will be a little away that other pipes so the birds knows that they can gain fitness by staying alive
# adding fitness since it has come to this level, also giving such little fitness value because the for loop will run 60 times a second so every second our bird stays alive, it will give it some fitness point, so this encourages the bird to stay alive
# every time a bird hits a pipe, we will reduce its score, so we are encouraging the bird to go between the pipes
ge[birds.index(bird)].fitness-=1
birds[birds.index(bird)].rip=True
# no need to add the below lines to delete the bird as, on hitting a pipe, we play the rip animation which turns the bird red and moves it to the ground and we already will pop the bird if it hits the ground so we dont need this.
# nets.pop(birds.index(bird))
# ge.pop((birds.index(bird)))
# birds.pop(birds.index(bird))
# this is checking if our pipe is off the screen so we can generate another pipe
ifpipe.x+pipe.top_pipe.get_width() <0:
rem.append(pipe)
ifnotpipe.passedandpipe.x<bird.x:
pipe.passed=True
add_pipe=True
ifadd_pipe:
score+=1
forginge:
g.fitness+=4
# distance between pipes
pipes.append(Pipe(550))
forrinrem:
pipes.remove(r)
# checks if the bird hits the ground or goes above the screen and avoids the pipe
Finally, we will import the NEAT config file located at assets/flappy-config.txt. This file contains the tweaks and values that the A.I. will use.
The configuration file is in the format described in the Python ConfigParser documentation. Currently, all values must be explicitly enumerated in the configuration file. This makes it less likely that code changes will result in your project silently using different NEAT settings.
To learn in detail about what configuration files are, visit → here.
Having ‘neat.StdOutReporter(True)’would give us detailed statistics in the terminal. Something like this:
Detailed NEAT Neural Network Statistics
"""
Code for importing and running the config file located in the assets directory
# now we will add stats reporters which will give us some outputs in the console where we will see the detailed statistics of each generation and the fitness etc.
