Everyone seemed to really love the video of Albert learning to walk, so I thought I would make a video improving that walk! In this video Albert starts with the brain he developed in the walking video, and he uses that knowledge to learn how to climb up and down stairs, walk on uneven terrain, drop off ledges and even climb up a backwards escalator! Albert is getting better and better:D If you’re interested in learning more about exactly how Albert works, I’ll do my best to explain it here in a simple (but correct) way, but this explanation is nothing compared to what you would learn going through the “Introduction to Neural Networks” course on Brilliant. Despite them sponsoring the video, they’re not paying me to write this comment, I’m writing this because the course is genuinely the best introduction to neural networks (like Albert) I’ve seen, and access to all their courses is completely free for 30-days if you use my link Brilliant.org/AIWarehouse to sign up for a free trial. The 30-day free trial would give you plenty of time to get through the relevant courses “Introduction to Neural Networks”, “Reinforcement Learning” and “Artificial Neural Networks” completely for free:) It's also a great way to support the channel, as you know it takes months to make these videos, and just starting a free trial using Brilliant.org/AIWarehouse would help me out so much! Anyway, onto how Albert works! If you're interested in training your own AI like Albert but don't know how, there's now a really easy way to do it! Luda, an AI lab, recently built a web app that allows you to create and train your own AI using deep reinforcement learning (just like Albert) completely for free in your browser! You build your own character (called a Mel) with lego-like building blocks then watch it train in real-time on their website in just a few minutes (really). It's an awesome project, and just like my videos, makes deep reinforcement learning so much more accessible, which is why I love it so much. This section of the comment is sponsored by Luda, but these words are entirely my own, it's an amazing project that I would have been obsessed with had they released it before I built Albert. I've genuinely been looking for a sandbox/game exactly like this since I was a kid. They're still early, but they're giving my audience first access to their closed dev build. Make sure you check out their site and create an AI agent for yourself!:D prealpha.mels.ai NOTE: It may look like there’s only one Albert training, but there are actually an additional 280 Alberts training simultaneously behind the camera to speed up the training time (and also reduce the amount of footage we need to sift through). HOW THE BRAIN WORKS I’ve explained this in previous videos, but Albert is controlled by an artificial brain called a neural network. His brain has 5 layers, the first layer consists of the inputs (the information Albert is given before taking action, like his limb positions and velocities), the last layer tells him what actions to take and the middle 3 layers, called hidden layers, are where the calculations are performed to convert the inputs into actions. Albert starts off using his brain we trained in the walking video, and we use reinforcement learning to further train his brain for this new task of climbing stairs. The only things that matter when it comes to deep reinforcement learning (reinforcement learning with a multi-layered neural network) are figuring out what information to give the agent (the observations/inputs), and when and how much to reward and punish it (the reward function). Almost all of Albert’s inputs are exactly the same as the inputs we gave him for the walking video (like the position, rotation, velocity, angular velocity, strength and contacts for each limb, plus some sensor observations), in this video however, we also have a sensor sticking out in front of each foot which tells Albert the distance between his feet and the stair in front of him, if there is one. These sensors are the only way Albert can ‘see’, and they only detect stairs, so Albert is essentially blind! Just like in the last videos, Albert was trained using reinforcement learning. For each of Albert's attempts, we calculate a score for how 'good' he performed and use an algorithm called proximal policy optimization (PPO) to make small, calculated adjustments to his brain to try to encourage the behaviors that led to a higher score and avoid those that led to a lower score. You can think of increasing Albert’s score as rewarding him and decreasing his score as punishing him, or you can think about it like natural selection where the best-performing Alberts are most likely to reproduce, so the genes that lead to better behavior will get more prevalent with each generation. As long as the reward function (how we reward/punish Albert) is set up robustly, these small adjustments to Albert’s brain over (sometimes) hundreds of millions of attempts leads to Albert getting very good at completing the goal defined by the reward function (climbing up and down stairs). REWARD FUNCTION Climbing up and down stairs is very comparable to walking so most of the heavy lifting for the reward function was done in the video where Albert learns to walk. In that video, I had to deduce exactly how and when to reward/punish Albert to get him to take steps, so for this video, I used that same reward function to encourage Albert to continue taking steps, just with one additional punishment; punishing Albert when he stands too still. I’ll get into why I had to add this reward in a couple of paragraphs. The reason we have levels for Albert that increase in difficulty is because if we were to throw Albert onto the escalator with only his knowledge of walking, he would have no hope of coming even close to getting through the course since he would need to learn many very complex motions to make even a little bit of progress. Also, I just made a community discord server! If you want more updates on Albert and learn more about exactly how he works (and just talk AI in general), make sure to join! discord.gg/jM2WkNuBnG :) The first level is as simple as we can make it; just a few single steps. These steps are designed specifically to make Albert trip, that way he’ll eventually learn to associate detecting the step with a new environment that needs a different behavior. Albert (and reinforcement learning agents in general) tend to learn to do whatever is easiest while still improving their scores. In this case, Albert originally realized it’s easy to just stop walking and stand still when he detects a stair, and when he does that he no longer trips and falls over the stairs, so just standing still is the easiest way for him to get a higher score. That’s why we added the additional punishment for standing still, we pushed him so he stops getting stuck in that local maximum reward. After doing that he was much more willing to try to step over the stairs and sure enough, after a bit of time, was able to do it consistently. OBSERVATIONS As mentioned above, we give Albert the same information we gave him in the walking video (limb positions, rotations, velocities etc.), except this time we give him 2 additional observations in the form of sensors coming out of his feet to detect stairs in front of him. Giving him these 2 new observations means I had to retrain Albert’s walking from scratch with 2 additional null inputs, then continue the training on this new brain, since otherwise there wouldn’t be any room on the input layer to give him the sensor data. Thankfully, since the walking behavior had a very carefully modeled reward function and I didn't need to record the training, it didn't take long for my computer to train it again from scratch (about 45 minutes). Adding new observations to this stair-climbing brain would be much more laborious since I would need to train each room at a time. While the walking AI didn’t actually need all the rooms to train properly, this one did. FINAL CHALLENGE The final room was very hard for Albert and unfortunately I ran into a big issue I didn't anticipate. I realized too late that since Albert is only given the distance to the ground under his feet and the distance from his feet to the stairs in front of him (if there are any), he isn't able to differentiate normal stairs and the escalator, but the motion required is very different, so it makes it impossible for him to only use one brain. I didn't plan on making him go up an escalator, that was an idea I had later on in the development, and because of that, in the final run he actually uses 2 different brains, he uses his main stair brain until he hits the escalator, then uses the escalator brain, then when he reaches the top of the escalator, I swapped it back to his main brain. There is definitely a way to have one brain work for the entirety of the final room, all that's needed is to give him an extra observation of whether or not the stair in front of him is moving, that would have trained the escalator behaviour into the main brain, unfortunately I just didn't have time to re-train everything to make this work, since it already took 3 months to make the video lol. We're looking to hire people to help make these videos! If you're a talented Unity game developer you can apply for a full time position here forms.gle/ko54z1LQmZNUT9Vp8 And if you're a talented AI developer (ML-Agents), you can apply here! forms.gle/Uou1Vwb5Q9VccaAY7 We're looking for full time employees, but part time works too, what we're really looking for are skilled and passionate people, so feel free to apply if you're interested! :D Thanks so much for reading and watching!

I cannot wait til we get to the point where Albert is put in an open world and is given a list of quests and adventures

Us: Robots won't take over because we haven't given them a reason to Also us: *Alberts in a lot of pain right now*

"If you beat everything, I'll let you out" DUDE YOUR'E GLADOS

This man did what no one else could He made a sponsored segment actually enjoyable to sit through

Fun fact, the binary code graffiti on the wall of the second stage spells out "help" Very humane AI living conditions, I see

I'm so invested in Albert's development Also i like the idea of a person just casually doing a backflip to not fall on the stairs

That final walk though, that arm swing just makes the music even better

I hold my breath every time Albert gets close to his goal. This is more suspenseful than TV drama.

I’m a simple man. I see AI warehouse posted a new video of Albert learning, I click like.

Can we get a "Albert learns to backflip"? That is clearly his dream, we would love to see it as (virtual) reality.

The fact "help" is written on the wall by the stairs is hysterical to me. Someone is sending Albert a message!

Two great things about this.... No annoying voice over. And good grammar. Both of those are overlooked these days imo. Amazing content! New subscriber here

Is no one going to talk about the simplicity and beauty of that sponsor placement? From the way it starts with the map of Albert's brain to make you think "What am I looking at?" right into the placement (with the lesson examples) of "You can learn this and more with Brilliant. Here's my code." Well done

I’m so proud of Albert! Figuratively, he’s gone very far. But in a much more literal sense he’s still that same little back-flipping orange box with vacant eyes. :)

I have been doing some research on reinforcement learning models for my AI classes and this video is a great excuse for me to enjoy the shenanighans of Albert in the name of research. I love AI warehouse ❤❤.

That video was so amazing! I was so happy to see a new video posted by you, I've adored all of them to pieces! And the way you showed your sponsor was the smoothest way I've ever seen any youtuber do it. Great job!

Albert inspires me to learn artificial intelligence so I can make a family for him.

I love how Albert discovered his love for backflips in the process... also this is the best AD integration I seen for my twelve years of being addicted to youtube

Thank you for adding the complete run at the end, that makes it all the more impressive!

I KNEW IT! This series had potential for some secrets and the binary on the wall in level 2 was perfect! I didn’t even catch it until watching it back a second time. Plus the ending was great too!

This channel rarely uploads and it takes alot of time and resources to get them done. Thank you for making these wonderful AI videos!

That's probably one of the most effective sponsorships I've seen on youtube. Had I not already taken university courses on the topic, I might be tempted to check them out myself. Very rare that Brilliant sponsorships do that to me.

I love how the narrator began this series so nice and compassionate to Albert and as time went on, they got progressively more critical and sadistic. It's a really interesting and clever kind of subtle storytelling, and I'm excited to see where it goes from here

Ive seen a LOT of Brilliant sponsors but I have never seen one like this where it just fits perfectly into the video where it just doesnt look like a cut in part and it caught me off guard. Very interesting.

"So Albert is in a lot of pain" Albert: *proceeds to die while flopped once again*

Love the sassy commentary, I can't resist watching another one of your videos when I eventually see one on my feed 😊

Your videos are so good! Music, editing, concept and execution! I couldn't think of anything you could do to make these videos any better, and i love the fact you can see Albert progress from cube to whatever the future may bring :D can't wait to see where this goes, subbed!

When he started hopping 2 stairs at a time at 11:22 I actually went "oh my god he's insane"

Albert is like a competitive gamer. The moment he feels any adversity, he just sends it and griefs all the way.

Episode 2834: Albert learns to care for children while his wife tries to divorce him

That sponsorship was brilliant putting it in the middle so it doesn’t interfere with the video

I would really enjoy like a five hour video of just training footage. With barely any editing, just the stuff you recorded for the montage anyway. Would be great for studying or gaming. Everytime there's a break, you can look over at how far the AI has come. Anyway, quality over quantity. Great work from you!

I've been here since the very beginning of albert's journey and it makes me SO PROUD TO SEE HIM IMPROVE! I think a really cool thing for him to try now would be learning to grab, push, pull and carry things! Maybe he has like a key he needs to take somewhere else or pull on a rope to open the door. No matter what he learns next I can't wait to watch!

It’s neat seeing how he swings his arms to keep his balance while he’s going down the really big steps at the very end. His improvements are very impressive!

I love that last stair set down to the green button. The way Albert jumps down it is so human hahahah. It seems like all the individual obstacles he was trained on helped with all the rest too. Albert ended up super stable on the final obstacle course

I loved the hidden "help us" message at 3:50 hahaha Nice easter egg!

8:44 when in doubt, backflip out

That could be the most well-put together sponsor section i have ever witnessed. I was engaged the whole time. Well done

I love how there's CC for this repeating all of the written out text in the video. Redundant UKposts is great.

3:24 I cant even backflip! Well done Albert!😂

I'm so impressed at this little guy's ability to fall down and remain upright! Seeing him walk from a first person perspective would be very interesting.

This is the "Saw" movies for AIs. AI Warehouse: "Albert... I want to play a game!" Albert:

That's the best ad implementation I've ever seen. And amazing video by the way.

That final walk was really impressive.

2:19 "It's only up from here!" They say, as a set of downward stairs are literally on the screen lmao. Anyways I'm such a huge fan of this channel, keep doing what you're doing.

The ad run was absolutely Brilliant!

This was thoroughly enjoyable! One thing I'd like to see, is the real time hours it took Albert to escape. He really was carving through those interactions!

I just want to say that I appreciate that you didn't interrupt the video with the ad. Keep up the good work and congratulations on the sponsorship!

9:42 - 11:37 Albert doesn't need to hear all this, he's a highly trained professional!

For some reason i got kind of emotional with that ending. It's absolutely brilliant, and also a neat idea of using ads in videos without it being anoying!

The editing on these videos is always superb

7:38 Looking at this FFW, looks like the staircase is going UP rather than DOWN xD As always, great work, for Albert making throught the obstacle course, and AI Warehouse for making it possible :) I wonder what will be Albert's next lesson.

It would be interesting if the punishment for falling were based on how much a human would feel from a fall, like the speed of impact and such, so he learns to catch himself when he falls

Magnífico. Sois increíbles, he pasado un rato verdaderamente divertido con Albert 😂 esperaré por el siguiente vídeo. Felicidades, es un increíble trabajo, ¡ánimo equipo!

I love the dedication you put into these videos. Keep doing like this :D

7:18 9yr old me on the mall:

10:22 I like how it can detect a square but it never draws a square

Рад что на канале теперь есть переводы на другие языки без автоперевода. Отличный канал!

Your videos are pretty cool, Im learning NN myself and its interesting to see how learning works from your videos. Hope to see some more of Albert's adventures :)

11:59 Bro got yeeted to space.

the only sponsor that doesn't ruin the video

That sponsor integration was pretty smooth actually, I liked it!

Sorry I’m a month late to comment but can I say, that I love Albert! He makes me smile and laugh when I watch the videos you make of him! Keep it up and hope to see more of Albert!

i still remember when little albert doed the worm break dance step, trying to walk... now hes chosing his hobbies... they grow so fast 🥰

Awesome way to integrate sponsors in a way that doesn't interrupt or impede the video in any way!! While also being prominent, visible and relating to the video. I love Albert, but I'm also impressed with how creatively you think.

Wow, is amazing, I´m doing my master thesis degree with ml agents... and omg i see your videos and makes me try harder and improve it as much as i can! really amazing videos.

I love how he developed this little arm swinging motion, I'm assuming as a way to keep balance 💚💚

Congrats on the sponsorship with Brilliant! This seems like a perfect partnership for your content so I'm happy to see your channel getting recognition for the amazing high quality content you provide. That sponsorship segment was incredibly well done and made me want to sign up for a free trial. Thanks for inspiring learning and curiosity :)

Albert’s pain is my source of energy.

Looking forward to the next video. Want to see him cycle. Also, maybe teach Albert how to backflip too? It'll be great seeing the progress and broken bones

I would love more than anything to keep watching this series of him learning to move across new terrains or manage other basic human-like motor functions. I want to see how human he can get

babe wake up new ai warehouse video are you one person btw? i've been seeing a lot of content farms but the way you upload it feels like you're one person

That music is fitting perfectly at 9:09 love it!

I really hope albert gets friend, rivals, and has a whole series with him as the main character. The Training arc goes crazy

That's the most brilliant ad placent I've ever seen. Haha, the only time an ad didn't annoy me 😂

"And they say *you're* the one being tested" is such a GLaDOS line

Your channel destined for success. The video quality, Albert's engaging journey, clever easter eggs, entertaining music, and polished editing are the reason why I'm able to sit through a 10 minute video of an AI walking. Not to mention the brilliant (I'm sorry) sponsor placement. Kudos on the great work - keep it up!

Glad this got another vid for Albert liking to backflip must have been the repeaters of walk with the push of other leg for walking

Wow, that was incredible. The way he was leaning forward and moving his arms in the escalator just like a human would actually do.

I really love how you can see the difference between his walking skills at the start of the video compared to the end. He's doing great!

I love how after a certain point, every time Albert almost fell backwards, he tried to do a backflip. I guess he learned that if you're going down, you gotta go down in style! ✨

I love your Chanel! Pls make more videos and quicker, I enjoy them soo much

Cmon where’s the Einstein in his name?

9:36 i was going to comment - training each oart separately is no good because theyll forget previous sections. Glad you included this section for everything.

First of all, that's one of the best sponsor message integrations I've seen: Just keep going with the content that just so happens to take place in the message Second, Albert is getting really good these days! Albert is walking really nicely in this one by the end. Still hoping to see some of the suggestions I've given in the past, like slippery floors. I think walking downstairs is generally harder than walking upstairs, so a downwards backwards escalator would also be a great addition. And slopes. Some randomization would also be good: It looks like Albert learned a more or less specific path over the rough terrain, rather than learning to deal with any rough terrain in general. So randomizing that terrain each time would presumably make him far more robust with that

Love the new editing style

9:00 Albert's steps sync to the song "Fine" by Lemon Demon!

I love how Albert learns and grows just like a kid

0:22 Stairs? Bro can fly

i wish you did it with the multiple different one's again. i like seeing how different looking ai react to the same situation

Great video!! It is very satisfying watching Albert learn Also, I just realized that with this letter font uppercase "i" and lowercase "L" look exactly the same, so our A.I. cubic friend's name could be ALBERT or A.I.BERT, and I think that is genius (even if it was not done on purpose)

Bro Albert’s flipping skills are elite 3:24 3:50

4:32 B A C K F L I P

You got some really amazing motions right there, on the scrolling stairs it looked like 110% human. I wonder if you could freeze the previous training, and concat new input with new trainable layers?

That was such a great ad for brilliant! 👏🏾

You should teach Albert how to jump! He really wanted to backflip in the video, maybe he'd like to learn jumping?

no alberts were harmed in the making of this video

I'm so emotionally attached to this little orange cube 😭 Albert my beloved

Albert my lovee😭 i was literally so proud