Everyone's commenting that its an excellent video but IMO this tutorial is awe-full! The instructor does not explain the process, hes all over the place going back and forth and just rushes through the concepts. If you want to blindly follow an online tutorial watch this video, if you want to actually learn the concept I would look somewhere else....

Thanks for the awesome video! Really fun to see agent improving during the games.

Thank you so much for this tutorial. I have always wanted to introduce snake with ML. and now looking forward to learning more about pytorch

A nice video. My only critique is that the presenter kept writing code for almost an hour without running it. That's not giving a good example.

I’ve always wanted to do this, thanks alot for the tutorial

Always very high quality videos when I just need it :) Thanks a lot! Looking forward to finish the video 😎

Excited to try this!

Awesome, so hard to find that type of explanation of dqn. All clear, great balance between theory and coding part for beginners in rl.

Watched the first 4 mins and the game and the learning process is fantastic! 🎉 Will go on it with the rest hopefully soon.

I did it!!! Thanks for showing this.

Super great video! I run it and after more than 2000 games it seems that the average score reached a plateau of 30. It Avoids boundaries very well. But it hit itself when the tail is too long. But this movement should be predicted before.

Very interesting tutorial. I was familiar with Snake code implementations, Tensorflow basics and RL principles and I could follow along. For some reason the plot doesn't appear, and I only get a small blank window where the plot should be. Probably some bug while typing the code down. Other than that everything works for me. Thank you very much!

That is so smooth bro, thanks

Thanks for this video, I have solved all my doubts

Thanks for freeCodeCamp for making this possible.

A video as valuable as a playbook👍🏻👍🏻👍🏻

Incredible work, thank you Patrick! PS: It is very funny to spot the typos/bugs before you do :)

Following this since I wanna make a pygame project of mine that I poured a lot of time into and have no idea what to make the game about play itself. Wish me luck.

Thank you for your great work. I couldn’t understand the equation you created in the video about the simplified Bellman equation, Q_new = reward + gamma * torch.max(model(next_state)). In this equation, model(nextstate) gives us a probabilistic action prediction. I couldn’t understand why we added one of the action probabilities to the reward. This is totaly different than the Bellman Equation. I would be very happy if someone could explain how the original Bellman equation was simplified in this way. Thanks in advance to everyone.

Very cool video! Highly useful!

Great tutorial

nice algo, but how to solve a problem with self destroy, when the closest cell to move is "inside body circle". I think a state must be all field with each body part,head, food etc, but its endless states and all of them unique and it will never learns or may be i wrong?

how to perform UCB, optimal initial values and dynamic programming approaches in this model?

Brilliant 🔥🔥🔥🔥🔥🔥

I think this project is awesome

Nice effort and seems like the convergence is achieved upto some extent!

This instructor is the best

Hello! Is there a way to save the state of the neural model? So I can load later a trained enemy AI, ready for being the player opponent? Thank you for the video!

You never fail to amaze us✌

Very good video. I need help with the homework. How can you avoid the loops?

Really interesting course !

Thanks for the video it was great but I have only one question if u can help after I run the agent and after a few try like 3rd game the game stuck and in the console suddenly start to count the number of game without the game really play

Lot of the code needs more explanation. There is a disconnect from theory and implementation. We copy all the parameters to this and to that without understanding why. The memory part is not well explained.

What to install before creating an environment ,iam confused

What about using GA to train the NN itself ? Will be a very interesting comparison no?

Theres no need of waiting a lot of time to train, on the game script you can just change the varaible from 30 to 1000 so snake goes much more fast and trains intself on less time.

This might be a stupid question, but how would one go about saving this trained model and accessing it for further training?

Really nice!

Is there a limit to when it stops learning? I mean the quality of the intelligence will stay the same at some point, or will it improve even more and more after those 12 minutes? Thanks.

Does this tutorial teach how to write "inference" by myself, not using library?

What's your VSCode theme? It looks gooood

i just want to know after training, how can we load back a trained model?

Is it reinforcement learning even if you give some instructions about the movements?

Thank you

Awesome!!

thank you very much.

I copied it code for code and it doesn’t work. It gives me no errors but just runs and then ends with no result🤦🏽‍♂️🤦🏽‍♂️🤦🏽‍♂️🤦🏽‍♂️🤦🏽‍♂️

Could you bring us a crash course of blender (3d modelling program)?

What are the prerequisites of the video ?!

very great course! thanks! A little question is , in class QTrainer maybe target = pred.clone().detach() ?

thanks, intresting!

The snake moves smoothly, but when it hits the first wall, the interface closes idk :/

38:13 "It has to be [0,0,1]"

So, my array for state is apparently returning NoneType. Anyone know a fix?

How can I get this terminal? :o

WHICH ALGORITHM IS USED IN THIS

So far so good... but how about load the old 'brain' if u exit the game and want resume later ? Oo iam bite confused tryd with state if os.path.exists('model/model.pth'): self.model.load_state_dict(torch.load('model/model.pth')) self.model.eval() in the Agent init. but that dosent work

true

thanks!

I want to modify the program: how do I make 4 outputs? I would like to integrate the snake length.

My snake became self aware, any ideas to stop it from taking over the world?

yes

What text editor

how to save the model...? where to know?

I'm a JavaScript Developer that knows nothing about python, but I must say I'm jelous 😭😭. This is really cool

I use your code and train it with speed 60000 (just modify the "game.py" file)

18:58 for me it is not showing as (pygame_env) ... .... i am a windows user (edit) it worked for I had to open a CMD outside of VS CODE and then follow the steps of conda pygame_env now I am getting "(pygame_env) C:\Users\........."

Can we speed up the game soo fast so ai can play faster and learn faster. Basically it can finish 100 games in a minute or two.

I think handing the fundamental flaw in the design to others as "homework" is a bit stiff... You're asking a neural network to solve an NP-Hard problem.

Found new hobby

Crazy.

why not just give the ai full board information as the state

where is he typing the commands? is this a CMD?

There is a small bug where the snake will eat itself if you go in the opposite direction that its going in. To prevent this from happening change this line of code: if event.type == pygame.KEYDOWN: if event.key == pygame.K_LEFT : self.direction = Direction.LEFT elif event.key == pygame.K_RIGHT : self.direction = Direction.RIGHT elif event.key == pygame.K_UP: self.direction = Direction.UP elif event.key == pygame.K_DOWN : self.direction = Direction.DOWN to: if event.type == pygame.KEYDOWN: # 2nd conditional to prevent snake from eating itself! if event.key == pygame.K_LEFT and self.direction != Direction.RIGHT: self.direction = Direction.LEFT elif event.key == pygame.K_RIGHT and self.direction != Direction.LEFT: self.direction = Direction.RIGHT elif event.key == pygame.K_UP and self.direction != Direction.DOWN: self.direction = Direction.UP elif event.key == pygame.K_DOWN and self.direction != Direction.UP: self.direction = Direction.DOWN

Cool vid 😊

can someone help me how to create a multiple snakes?

so i had the program run over 2000 games and it couldn't get past the 80s mark. how would i get it to improve past that?

someone tell me how to save trained data?

omg patrick wow

Hi I have a question. I followed along and wrote everything. When I run it in the terminal it works but the plot doesn't. It gives me an empty minimized white screen. If someone else experienced this please help.

Wow

Can we create AI bot to play dice game?

no windows opened when I ran. not even any errors. how do i fix it?

after 6000 games it averages at 33.4

somoone knows hot to imporve the snake so he does not collide with itself and loop over itself?

Jobs Is alive?

Grüße aus Deutschland

I want it to play minesweeper

I got this error: ValueError: not enough values to unpack (expected 3, got 2) for line 118 in agent.py

Do you legally own the AI you trained?!!

Very interesting topic, but a very bad teacher. He doesn't explain almost anything and the little he explains he talks about them very briefly like we already know all of those things and we make a revision.

What was the theme?

dumb question ik, but what is the box he is usint to code, is it CMD, or windows power shell etc ?

Not first 😑

12:00

16:51

5:31

set video speed at x1.25 or x1.5, thank me later.

The training time is very very long

0:00 white man jumpscare