New meta for your next speed run? But seriously, your explanations of speed runs for the game helped me a ton in understanding some of this stuff....even though I still got some of them wrong.

@@Chrispresso unfortunately with the walljump in 2-1 you can only get 6 on the TIME remaining so you get 6 fireworks :( I'm really surprised about the 7-1 fpg it did, since what it did there would actually waste a lot of time lol. No problem! Great vid!

Nice, kosmic watched this!

tbh i accdently did a wall jump on a pipe un 1-1

Gaming

Mario's greed ultimately killed him

Shiny, but deadly coins

I usually don't like comments but when this one said 99 I just couldn't help it.

@@easyaspi31415 I rate this comment an unbelievably exceptional 5 out of 5 fuzzies!

@@veggiet2009 we don't have royal fuzzies, and we don't have rainbow fuzzies. How about 5 *BIG CHUNGUS FUZZIES!?!*

Maybe wall jumps are just easier than running. Maybe I've been playing wrong...

@@Chrispresso Well if it can only see three types of things, -1,0,1 then running limits interpretation time and leads to errors and death. Wall jumps are done on already interpreted object encounters so from its point of view I would say wall jumps are easier. Running is just too risky.

Because coins can be an indicator of when to jump I kept them in. Ultimately it may have been its downfall...

@@Chrispresso you could've maybe made it see coins as a separate thing, similar to how enemies are seen separately from blocks

@@laurinneff4304 ”The reason I did 3 was so I could give normalized values (-1, 0 or 1) in this case for the blocks. If I wanted to use more than the three I needed a way to encode the inputs (one-hot encoding or something similar) but I was running this on a pretty old laptop at the time and wanted to simplify the inputs. ” - Chispresso

hi darby!

A wild Darby has appeared.

Hi BTG!

Thanks :D Glad you enjoyed! Ya... that section was just tricky in general unless you can either slow down or know that jump is coming. There are techniques that could get around that like using a memory. Maybe next time!

@@Chrispresso I'm thinking that maybe... have an extra backpropagation, rewind, instant replay type mechanism for the last couple of hundred frames only when he dies... This would be different from the long term memory of learned traits... This way, when the A.I. runs through a level and dies... upon death, the last few hundred frames or approximately the last 3 - 5 seconds of gameplay is recorded into a buffer net only when the death condition is met, and this is replayed through its own mini-neural net to be trained upon... Then these become learned traits upon the next trial... You may have to extend that frame count or time interval depending on the nature of the death... Was it contact with a creature, or was it from falling into a pit... if falling into a pit, then it depends on the height and time taken for the fall. You would want to rewind back to where he was at least a solid second or two on the previous stable platform... That and you may need to add in a dynamic look-ahead feature too.

OR anyone else for that matter. Thinking back on my childhood ;)

Yep! Very good observation. The reason I did 3 was so I could give normalized values (-1, 0 or 1) in this case for the blocks. If I wanted to use more than the three I needed a way to encode the inputs (one-hot encoding or something similar) but I was running this on a pretty old laptop at the time and wanted to simplify the inputs.

@@Chrispresso DO it again with the right amount of inputs pls :D I'm super curious off AI limits. For example, if it thinks it found the fastest route, will it still look for a different even faster one. And will the AI ever discover he can go down the pipes ever? Is there a way to make it look for the highest score in the fastest way possible. Man I love AI, thank u for the video, I subbed btw ;)

@@Chrispresso You can make it a CNN, with multiple binary channels representing different types of objects.

@@Corpsecreate That's true. I might have to try that since it should use less weights overall as well.

Wow! That means a lot! Thank you! I definitely want to make sure the content stays beginner friendly and have been thinking about including some bits of code in the videos but haven't been sure. Also considering doing some stuff with Unity to get some eventual better graphics and environments for AI.

Happy to hear that!

Ya, I was very surprised that these AI were able to learn some of those tricks!

@@Chrispresso I am impressed with the breakthroughs that your ai's have been able to do. You should make tutorials for this stuff.

i am skeptical. using the algorithm time loss is not (in my observation) weighted heavily enough to merit the discovery of those glitches. those glitches save fractions of a second.

@@johnforrestboone1 THIS and it took years for some of the greatest Mario players to (kinda accidently) figure out these glitches and it also took a huge amount of game knowledge to come up with something like forward dash. Speedrunners were saving a pixel of forward movement cuz the world updated every 20 frames instead of every frame or something like that and this AI has what 10k generating of knowledge of smashing keys lmao

Noted!

@@Chrispresso wow, thank you for taking the time to read my comment. I look forward to watching more of your amazing videos.

Welcome! Glad you're at least being recommended things you're interested in! Definitely will revisit Mario in the future.

Thanks! I'm glad you're able to enjoy it! I know what you mean. I found myself watching way longer than I should have at times....

did it beat the game?

@@TWFDeadzoneII It just finished level 6-1 and I'll be going to 6-2 soon. So far it's beaten every level except for maybe one or two. I think world 2-2 is one that it had some trouble finishing. It would get through the pipe at the end but had trouble going all the way to the flagpole at that point. For the most part, it's been great at figuring out the levels.

@@Scn64 that's cool. Have you made any changes to the original code?

@@TWFDeadzoneII I've made some changes in the config file just to test different strategies. The default strategy that the creator put in still seems to be the best though. I've also made some minor changes to the Python code that just affects how some information is displayed to the user...the core functionality is still all original though.

Thanks!

ceave gaming

@@eloygarcia9221 game ceaving

Thanks :) Definitely working on more. Just takes time to build some of the AI. Hoping to get onto a more regular upload schedule in the future!

Glad you enjoyed it! And I'm happy it wasn't boring or overly complicated :D

Haha I'm not there yet... but maybe some day :D

@@Chrispresso Oh sorry. I'll come back in 4 months. Should be applicable then.

Glad you enjoyed it

Glad you enjoyed it and thanks for the sub!

Oops, sorry for the mistake, but thanks for the clarifications! Guess the AI knows more about it than me...

Glad you enjoyed! I will definitely try to add that in future videos :D

Same over here, just found you and id love to see the progress

You can make the pink box as large (tall and wide) as you'd like. I made the code pretty flexible with it. I thought about that but didn't end up adding it. Even if you use the blocks but pooled them into 2x2 for safe/unsafe/empty it could potentially help. Might end up trying that or even just try using a CNN.

It would be! Maybe some day it will. And thanks for that. I thought it was just the one pixel, but I was just reading about it from some speed running websites and probably just misunderstood it.

@@Chrispresso Its actually a very common misconception that there is only 1 pixel you can land on, and not a lot of people know its actually a 5 pixel window, so places you read might have actually been saying its just a 1 pixel window lol. For example, this bismuth video about SMB3 compares the smb3 wall jump to the smb1 wall jump (link is time stamped), and this video says that smb1 WJ has only a 1 pixel window, and bismuth didn't know otherwise until I told him ukposts.info/have/v-deo/bIeggaJknXuB1Zc.html

I thought about this but it wasn't an issue early on. By the time I realized I should have done it, I'd spend so much time training the current AIs that I didn't want to do it over.

@@Chrispresso hmm in a different version?

The AI figured out how to do it before I even really knew what it was....

@@Chrispresso "ah crap I wrote a bug into my code" speedrunners: "OMGWTFBBQ IT'S SO GOOD. DOUBLE FLAG POLE + INFINITE WALLJUMPS"

Ah, so that's what LuigI/O is! Just looked up the channel and that's pretty cool! They were different neural networks. They used the same base from previous levels but could change if they needed to. So the weights from 1-1 carries to 2-1, then 2-1 to 3-1, etc. Ya I don't think wall jumps were intentional, but once it knows it, it's pretty easy to do again. Since it's able to see ahead for the blocks, a lot of times it would do a small jump into a wall jump. Obviously it only works under certain conditions though since the weights are unique and not shared like in a CNN. I still need to test on other levels like -2, -3. Originally I had planned to but the states I needed for OpenAI gym weren't there and only included the -1 stages. I do plan to come back to this in the future with a more advanced AI and see how it fares compared to this AI and finally test it on all stages (-1, -2, -3, -4).

ah, okay the weights just carried over, yeah that's what ours did as well. But, it's impressive you were able to beat up to 7-1. 8-1 is an extremely hard level since it's the longest one in the game. I never thought about the coins fooling it since they look like blocks to the input viewer, but that could certainly be a reason why it screws up there. That would probably be pretty easy to fix. Definitely a lot of cool things you can add to try and improve this!

@@electra_ I imagine there will be a few levels that will give it difficulty when I try again. Anything with warp pipes might be a problem since I'm not sure how to encode some of that information or what it will look like to the fitness of the individual at that moment. Definitely excited to try out some new techniques though!

only level with warp pipes that really matter is 8-4, and that level is... basically impossible for AI lmao there's both land and water movement which work totally differently, there's pipes that you have to just know which one to go down exactly, there's a section with a required hidden block (or wall jump) and it's long. I believe LuigI/O needed 3-4 networks split up over various points to beat it at all, as well as a custom fitness function to tell it when it was on top of the high pipe, and a penalty for whenever it passed the pipe it needed to go in.

Glad it's inspiring! All my stuff is always freely available on GitHub. I've got a few other things currently in the works, so you should always have some stuff to follow along with :D

@@Chrispresso awesome man, got the notifications on for your channel, can't wait to see some more :)

Ya I saw MarI/O like 4 or 5 years ago or something like that and it's actually one of the reasons I first wanted to learn about AI. They are very different projects from what I can tell. MarI/O uses something called NEAT which basically allows the neural network to change architecture. I instead did a fixed architecture with a deep network and had a genetic algorithm control the weights of the network. NEAT has similar attributes to a genetic algorithm but not quite as good (my opinion) since it has to build the network from the ground up and often doesn't work well without re-training from one environment to the other. From what I can tell, MarI/O allows any box on the screen to be considered for input. I restricted mine to a certain view since I used a dense (fully connected) neural network. I'd say the biggest similarities are NEAT and genetic algo + neural network both rely on generations of improvement and they both use boxes of pixels as inputs. I also carried weights over between levels to help advance the next populations and to carry over some information like danger of enemies. Hopefully that helps and isn't too confusing!

@@Chrispresso very cool, I could tell that it was an improvement, as It seemed to be adaptive, though I couldn't quite tell why

@@Chrispresso is reinforcement any better than genetic?

@@abrasmage Hard to say if it's better or not. RL algorithms like A3C and PPO have seen a lot of success in video games though. I would say the main advantage of RL is that you let it learn even more by itself and let it figure out what it cares about.

Don't worry. Mario will come back with a vengeance.

He probably ran example runs with graphical output after X% clear rate.

or did you do that and that is the reason you only did over world levels

I read values from RAM for the emulator. You can check out the source code for more information.

I just give it the current frame it sees. This AI has no "memory" and no experience replay buffer. It doesn't have a concept of velocity necessarily, but it might figure out that being too close to X while holding Y is dangerous. So it may end up releasing X. That could cause it to slowdown if the buttons correspond to running towards the right.

That would be cool! I'll look into some of them! Side scrollers are a good environment for this type of AI.

This one does in fact read from the RAM, but you can definitely read from the screen instead!

Haha it might still never hold a record, but at least it was able to learn a lot.

I just wanted a slightly larger exponent for rewarding distance than penalizing time. I also wanted both to be exponential to more greatly impact large differences between values.The actual values of 1.8 and 1.5 were somewhat random, but the difference of .3 was chosen by graphing potential distances/times and ensuring that the fitness would continually increase and not go negative.

@@Chrispresso Okay, you got the difference by plotting a graph. To plot this how many simulation you ran (How many data points)?

@@SUNILKUMAR-og4vm I didn't run any simulations with the AI for that. I just took the total distance of the level and gave different random times it could probably finish in. I just wanted to make sure that if the AI finished the level in 20s and another AI finished in 40s, that the fitness between the two would be quite large. So I just created a bunch of fake data to see what it would look like.

@@Chrispresso Nice approach, thank you for sharing. Can you suggest any resources to read about developing and optimizing a fitness function.

@@SUNILKUMAR-og4vm I actually can't. I have mainly learned through trial and error and I know there is research being done in that area. I would say just think about what you really care about and make sure not to add bias. Like I could have rewarded "enemies killed", but then that doesn't weight equally between levels since there isn't the same number of enemies. It could also add unwanted bias toward certain sections of levels where enemy spawns are substantially more. A rule of thumb for myself is to think about the end goal and reward the end goal. Anything in the middle is bias. Chess? Reward winning. Don't reward capturing a piece, otherwise you introduce bias to the AI. If capturing a piece is important, the AI can figure that out. Same applies to a lot of different scenarios.

Definitely! I'm not sure how long it took people to find out about the flagpole glitch or wall jumps, but it didn't take too long for this AI. So it's very possible to use it to find other glitches in all types of games/environments!

Automatised runs are already a tool to find bugs in the classic development process. But, usually the studios don't use AI for that

@@Chrispresso This is already used in celeste classic, with a few python scripts and some restrictions, it's not really an AI, just brute forcing with restrictions

Three things 1) This game is already *super* optimized. It is possible, but unlikely in my opinion. 2) There is such a thing as a TAS, which is when a human inputs inputs for a computer to execute. So I guess that's like AI. 3) @Chrispresson I'm sure that wall jumps were ran into during development or at least the wall jump pixels.

@@relic374 a tas is just a predetermined set of inputs given by a person. it's nothing like a self taught ai. a tas isn't learning how to beat the game on it's own.

All my code is on GitHub so you can check it out in the description. I just parsed the RAM addresses for block types (easier for NES, but much harder for N64). The neural network is drawn with just a bunch of lines and circles. You could do something similar for Mega Man. I haven't played it in forever but most platformers could follow a similar approach to what I did here. Maybe I'll look into Mega Man a bit more....

@@Chrispresso I've been thinking about a megaman version of this for quite some time, but I've been stuck on how to get the neural net to recognize the game state. I don't much know anything about reading the rom data, especially when my vision includes using data from the sequels.

It would probably need to use something more advanced that this type of AI. This AI doesn't receive feedback until the end of an episode. For something like Super Mario 3D it would need to update as it progresses.

Ah, good question. The 10 additional inputs are a "one-hot encoded variable" That means that at most one of those 10 can be "active" (have a value of 1 in this case) at a time. The reason for doing this is to know which row of the pink rectangle it's in for the input. You give it a one-hot encoded variable to get rid of potential bias. So you're no more likely to think being in row 1 is better than being in row 2 or 3, etc.

@@Chrispresso So they're basically just a height indicator? "Which row of the screen is Mario in" In that case, what exactly does the gray/blue state represent?

That was hardcoded. I wrote a portion of code to read the RAM values and compare what they are to determine the type of block and then just grouped them accordingly.

Do you expect a plumber NOT to inspect a pipe???

Don't worry. Next time it will