How to Create a Neural Network (and Train it to Identify Doodles)
Переглядів 1,807,886
День томуExploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.
Source code: github.com/SebLague/Neural-Ne...
Demo: sebastian.itch.io/neural-netw...
If you'd like to support me in creating more videos (and get early access to new stuff), you can join my patreon here: / sebastianlague
The data is from:
Digits yann.lecun.com/exdb/mnist/
Fashion github.com/zalandoresearch/fa...
Doodles github.com/googlecreativelab/...
Cifar10 www.cs.toronto.edu/~kriz/cifa...
Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Biases
6:45 Hidden layers
7:45 Programming the network
9:57 Activation functions
12:42 Cost
15:07 Gradient descent example
18:22 The cost landscape
19:55 Programming gradient descent
21:10 It's learning! (slowly)
23:21 Calculus example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Digit recognition
43:56 Drawing our own digits
47:37 Fashion
48:25 Doodles
52:00 The final challenge
Music:
Cosmic Waves - Michael FK
Amber - The Stolen Orchestra
Beyond the Horizon - Sounds Like Sander
Air - Assaf Ayalon
Purest Form - Sounds Like Sander
Hear Wide Open - Sounds Like Sander
Universal Wonder - Moments
Roman P - Moments
All In Good Time - Shimmer
It Will Come Back - The Stolen Orchestra
Frontier - Shimmer
New Moon - Cloud Wave
Sunflower - Cody Martin
Inner Peace - Moments
Enchanted - Cody Martin
Just Around The Corner - Shimmer
@SebastianLague Рік тому
Hey everyone, at 20:17 it should be -= costGradientW (the minus sign is missing). I somehow managed to delete it while formatting the code for the recording! Thanks for letting me know in the comments.
@multiarray2320 Рік тому
this video is even a better explanation of NN than the video from 3b1b and i thought his video was already perfect. thanks for creating this video because now i'm able to understand it much better.
@shahzadansari849 Рік тому
I was Always Waiting for Your Next Video , Finally Something to Watch and Explore ! #thanksForVideos its really helpful explanation with real examples
@lucasgaperez Рік тому
youre so underrated buddy
@hiihkumar9633 Рік тому
@@multiarray2320 ez
@SaleemKhan-qc1lh Рік тому
Fac card so c
@TAP7a Рік тому
"I'm bad at naming things" There are only 2 hard problems in computer science: cache invalidation, naming variables, and off-by-one errors
@bingusdingus8239 Рік тому
Segmentation faults 💀
@rube9169 Рік тому
I see what you did there XD
@NStripleseven Рік тому
dang
@hhhpestock951 Рік тому
Truth.
@Temari_Virus Рік тому
0, 1, 2. Thank goodness, I thought there was an off-by-one error there
@niceguysayshi5765 Рік тому
As someone who has some experience with machine learning i can say this has to be the most intuitive explanation i have ever seen
@simonsemmler9804 Рік тому
indeed. this is amazing
@arivanhouten6343 Рік тому
That is exactly what I was going to say
@ProXicT Рік тому
As someone who has zero experience with ML/NN, I second this. What a great intro into the NN topic!
@benamende5897 Рік тому
The only thing missing is matrix math (I find it much more intuitive for some reason but I may be weird)
@beraulgd3662 Рік тому
Completely agreed! He actually walked through EVERY STEP (I can think of right now)! Which is great! One long video is soooo much better than scowering for dozens of bad ones (from experience)!
@wlockuz4467 Рік тому
I am a software engineer and I've always wanted to learn machine learning, being able to code is not a problem for me but being terrible at maths and statistics makes it hard to get accustomed to all the terms and concepts. I've tried multiple courses from different platforms and instructors and all of them try to teach you "what" to do instead of "why" to do it. I personally find learning more intuitive when I know why am I doing something instead of blindly following steps. This video is exactly the type of introduction to ML that I've always wished for, The explanations are highly intuitive and most importantly visual, there are no assumptions and no brushing over concepts, Nothings being done just for the sake of it, Everything is explained in simple language. I admit I'll still have to watch this video a couple of more times to make full sense of it because its jam-packed with so much information. You'll probably miss this comment in the sea of other comments (although I hope note) but I genuinely want to thank you, This video has relit my once dead interest in ML, I would love to see more videos from you on this topic or least get some recommendation on where I can learn more.
@SebastianLague Рік тому
Thank you, that’s wonderful to hear!
@AkshatSinghania Рік тому
Exactly what i wanted to say , This is the first time , I actually got to learn about the math behind machine learning and not just working with some ml library and get to know the true essence behind it. Great videos as always
@hepcat93 Рік тому
The story of not liking doing something just in the sake of doing this very something without thorough explanation is literally my school story with trigonometry and derivatives :D I couldn't understand it back then, but no one cared to explain instead just telling me to focus on the process itself.
@pascalschopper2817 Рік тому
Exact same situation here. Coding is fine. Maths....not so much. Certainly a bit of a lack of talent but school wasn't too helpful with the usual attempt of explaining the what without the why. @Sebastian, your videos are by far the best source for many of the topics i'm engaging in.
@pixboi Рік тому
My exact sentiment also!
@tartarugabradipo6081 Рік тому
Sebastian: "The End" Neural Network: "Oh, that's for sure a tractor"
@rienkthegamer5422 Рік тому
I’ve never had someone explain calculus so intuitively. The quality of this content is absolutely incredible.
@milanstevic8424 Рік тому
He's not underrated, his content is stellar and known in the community for quite some time. He has almost a million subscribers, and everybody knowledgeable in Unity forums knows about his channel. Besides, in my opinion, after years of seeing all kinds of videos on game dev, Sebastian's are still one of the best, even old ones. And I'm not talking about just the style of presentation, I mean this strictly technically, his advice and his code are next level. Also I'm not young (as a human or as a game developer, the age is similar), and if I can learn a thing or two from him, and occasionally I do, that's instant five stars from me. Maybe he could become even more popular (though popularity comes with its own pressures and costs), but he's definitely not underrated.
@johnhadden3998 Рік тому
its pretty surprising that they don't teach differentiation by first principles in some places
@rienkthegamer5422 Рік тому
@@milanstevic8424 I am very sorry for abusing underrated in this manner, it’s just that, in my personal opinion (which is often wrong), concepts in his videos are often explained so thoroughly, yet he makes is so easy to understand, that I don’t see why more people love his content. And I know that CS isn’t for everyone, but *just* (I’m using just very loosely here, this is no small accomplishment) 1 million subscribers for videos unrivalled for this amount of quality is, as I said earlier, in my opinion, underrated. Sorry and I hope you understand, I might still be wrong Edit: to avoid confusion, I have removed underrated from the original comment
@milanstevic8424 Рік тому
@@rienkthegamer5422 Maybe I'm just picky with the words. This channel is definitely not as popular as, for example, Brackeys (although that channel is officially dead for a year or so). But to say it's underrated is something else. Maybe you thought like underappreciated by the general population / not recommended enough by the UKposts algorithm, and I can agree to an extent. But let's ponder instead, whether such front-page popularity a good thing. Those who do got to see his content, would definitely recommend it, and in that sense it is not underrated. His channel is constantly growing, regardless of the algorithm. I've seen people recommend Seb's videos to people who cannot grasp even the most fundamental concepts, so that's why I've said that popularity isn't always the best thing. His videos and experiments are very smart and sometimes beautiful, but you still want the audience to understand the beauty and the effort behind it. If you cultivate a wrong culture here (for example, children screaming that they get errors for the most mundane things and nobody understanding a word of what he says) his videos and his enthusiasm would certainly be affected by this. So, with all that in mind, maybe "underrated" is a good thing? I definitely would not call 1M a bad subscribership. That's gold on UKposts. Btw, now that you changed your message, I agree 100%. I'm probably just picky with the words.
@khiemgom Рік тому
Well then prob u dont watch enuff maths related vid, there are other talented maths teacher on youtube, saying like that is an injustice for them
@saar5947 Рік тому
You're probably not gonna see this message, but I want you to know you give me inspiration and motivation to learn, do and achieve more as i believe you do for many others
@SebastianLague Рік тому
I’m very happy to hear that, thank you!
@zeddoes Рік тому
True. He is the only channel I turn thr bell 🔔 on. It’s always worthy to watch the video
@exildur Рік тому
@@zeddoes Same here, I'm subbed to close to 100 channels and this is the only one I have the bell notification on.
@mulualemtekle6094 Рік тому
my life would be very different if I hadn't came across his coding adventure playlist, I got into game dev and computer graphics because of Sebastian and I fall in love every day with the beauty of things in this realm
@Malizma333 Рік тому
I love how this channel focuses on quality over quantity, shame the youtube algorithm doesnt promote more of that
@CYON4D Рік тому
The production quality is off the charts. As a software developer I can't even imagine the amount of hard work, research, technical knowledge, expertise, patience and determination this must have required. Hats off to you :)
@SebastianLague Рік тому
Thank you!
@Suburp212 Рік тому
I wish somebody had explained calculus like this in school. Intuitive, descriptive, visual, simple, elegant. This content is marvellous.
@thehollowknerd3858 Рік тому
School's advertising is such a complex topic, but it's simply because they don't teach it well. I was able to learn it in eighth grade while I was struggling with 10th grade math. Schools are bad at teaching but don't let that hold you back from accomplishing what you want.
@TheBoglodite Рік тому
Mfw Sebastien tricked me into learning calculus
@JustinKoenigSilica 6 місяців тому
We did get taught almost exactly like this in school. Most people didn't get it anyway.
@mangoalias608 Рік тому
everything else aside, can we all appreciate the incredible visuals all of Sebastian's videos have? the animated graphs, the visualizations, the explanations, its so pleasing :)
@senftube2460 Рік тому
Yeah, Its so smoooooth 😍
@BlenderTimer Рік тому
I certainly appreciate it! Making good visuals is not easy!
@BlenderTimer Рік тому
@@jitin4179 He shared a link to the source code on his community page.
@To-mos Рік тому
I'm guessing it's a Unity3D framework he made for doing visualizations in the engine.
@superturkey2458 12 днів тому
@@To-mos Only Losers call it unity3d.
@The14Some1 Рік тому
25:18 OMG, and at this point I've completely realized the true nature of the derivative - why it becomes a slope function, why x^2 turns into 2x and so on. It was one of those mind-blowing moments of insight, which most of us have experienced at least once in our life. Thank you, Sebastian!
@ET-yc4wb Рік тому
I haven't even learned calculus and this was my first introduction to it. To my surprise, I understood it perfectly and I'm probably gonna start self studying calculus before I study it in school.
@lordleo8563 Рік тому
same
@To-mos Рік тому
The word you are looking for is epiphany.
@The14Some1 Рік тому
@@To-mos Ah yes, thank you.
@n3ttx580 Рік тому
I had the same when he was just going about adding and multiplying many values together, when it struck me; addition (and thus multiplication) is computationally A LOT less expensive than division, so adding/multiplying tens of numbers can be as fast if not faster than division. That's probably the reason behind all the ridiculous math - to speed up everything by doing what computers are best at: addition, so it can be optimized (in terms of algorhytms) furthermore without major performance hits.
@GumRamm Рік тому
Any reasonable person: “I will use one of the many existing Python libraries that implement backpropagation for me in an efficient and easy-to-use way” Sebastian: “I like writing C#” Jokes aside, very cool and impressive project as always. And of course presented in a stunning and intuitive fashion, keep ‘em coming!
@wlockuz4467 Рік тому
“I will use one of the many existing Python libraries that implement backpropagation for me in an efficient and easy-to-use way” I hate that its like 99% of online courses
@weckar Рік тому
The reasonable person learned nothing.
@notahotshot Рік тому
@@weckar The reasonable person learned you don't have to reinvent the wheel every time you want to build a wagon.
@weckar Рік тому
@@notahotshot pretty good to know how a wheel is made for your first wagon
@kazioo2 Рік тому
@@notahotshot That's great until you need to heavily customize a wagon for a new kind of task, but those old wheels somehow don't work well with it and you have no idea why, because you never learned how these wheels even work. This is a very common problem in programming in recent years. There are so many programmers now who never learned basics (eg. they often don't even consider they operate on physical hardware with actual memory), new apps with the same UI and functionality we were using 20+ year ago start lagging horribly on a 10,000x faster computer than what we had back then. Oh and you better have a few GB of memory for those nice fonts...
@blitzarsun Рік тому
This is the most intuitive explanation of machine learning I've watched. I hope you return back to it soon!
@oskar7063 Рік тому
This video is looking awesome, do you have you're own animation library like manim or the one from aarthificial or Freya Holmer?
@SebastianLague Рік тому
Thanks! I have my own library, although that’s a generous term to be honest, it’s more like a loose collection of scripts scattered across several projects (which I always have to hunt down and repurpose for whatever I’m working on). Really need to improve that some day! I have recently been experimenting with Freya’s (brilliant) Shapes plugin for rendering lines and points though, so some of the graphs and a few other things in this video are using that.
@idophysics0113 Рік тому
@@SebastianLague Been a huge fan for a long time! Would you be able to share more about your library? I've used manim a lot but I'd be very interested to see how you create your animations, even if you only show a small bit of it.
@FlyingDominion Рік тому
This video has so much animation. Did it take longer to code the project itself or make the video?
@timothysnave Місяць тому
Holy crap that car in the beginning even found like a line of best fit. Crazy.
@nocturn9x Рік тому
I began loving this video the second I realized you had explained derivatives without actually mentioning them. I love practical approaches!
@nrub Рік тому
Ok, the way you explained derivatives at 25:00 makes soooo much more sense. It perfectly presents why first derivative of x^2+ax+b is 2x+a. I'm now so angry this wasn't explained to me properly when I was in university. I also believe you are secretly a maths professor.
@jamesmnguyen Рік тому
I would argue your Highschool teacher should've taught you derivatives. Unless, however, you took basic math in Highschool (which isn't a bad thing)
@KnowledgePerformance7 Рік тому
@@jamesmnguyen not everyone went to school in the same system you did friend
@jamesmnguyen Рік тому
@@KnowledgePerformance7 I understand that.
@jonsmit8134 Рік тому
You can keep at most 4 new things in your brain at a time. Perhaps your lecturer explained tangents, secants, slopes, limits, derivative limit definition, specific derivative calculations, derivative dx notation, derivative prime notation, ect in too few lessons. Now that you are familiar with these things it is much easier to follow.
@multiarray2320 Рік тому
i dont think he is a math professor because he can actually explain it xD
@cvntdav Рік тому
I love every single video that you make, you make such amazing stuff! I wish you could bring back the "How computers work" series which made me discover my passion ★
@SebastianLague Рік тому
Thanks, I'm happy you enjoy them! Might be a little while before the next computers video, but I do still have plans for that series!
@cvntdav Рік тому
@@SebastianLague 😄
@dkaloger5720 Рік тому
I also really liked the how computers work series .It is definitely one of the fundamentals of computing I knew the least about ,along with many other people I assume .
@MrMcbram Рік тому
In the mean while, if you havent already check out Ben eater's video's! He has some great explanations from transistor level logic gates, to how these can be used to create all building blocks of a basic processor and how it handles assembly code
@JordanMetroidManiac Рік тому
Sebastian Lague, your videos might go down in history as the most well-produced educational videos on UKposts of all time. And it’s hard to even say that something beats 3B1B’s videos.
@SamirPatnaik Рік тому
yes but there is another insane, underrated guy who, imho, exceeds Seb in terms of production value: braintruffle
@jaideepshekhar4621 Рік тому
@@SamirPatnaik What videos does he create?
@codinghub3759 Рік тому
@@jaideepshekhar4621 From what I got from a quick search, it seems to be focused on simulating stuff
@bladekiller2766 2 місяці тому
@@jaideepshekhar4621 he creates videos that no one understands even PhD levels struggle with his explanations, so it's everything about the production. His videos are very very high level Physics.
@RabbleRousy Рік тому
As someone who watched a lot of these videos when writing my Bachelor's thesis on NNUEs (a specific kind of neural network for chess), I can safely say this is the best introduction to neural networks I've seen. Absolutely love all the visualization, how you start from the ground up but still include the calculus etc. Fun fact: my thesis was somewhat inspired by your Chess Engine video as well. I love your content, becoming a patreon now!
@SebastianLague Рік тому
Thank you!
@oinotnarasec 11 місяців тому
I wanted to write my bachelor thesis on chess algorithms, but my professor told me it was too complicated… would you mind sharing your work? I would love to check it out!!
@hulmaji1695 Рік тому
I cannot believe how lucky I am (how ALL of us are) that you sir exist. This quality, effort and precision put into these videos... Just wow... Thank you!
@avwie132 Рік тому
Having worked extensively with neural networks some 10 years ago, I must say this is hands-down the best explanation I have seen for people who are new to it. Excellent visualisations and explanations. It is so great for someone to start working on the absolute basics (simple perceptron) and working up, instead of directly going to PyTorch of TensorFlow.
@techyte8810 Рік тому
Sometimes I forget this guy is real and not just a voice in my head teaching me everything
@gpazsilva Рік тому
It's awesome how, after even having classes in college about neural networks, I finally understood how they work *in practice*. I studied the theory and saw a lot of "for dummies" explanations about NNs, but people usually abstract the actual code from their explanations and this used to frustrate me a lot. Thank you so much for the explanations, Sebastian; your content is gold.
@vb0t429 Рік тому
It's impressive what Sebastian can achieve with enough time and dedication
@thephoenixsystem6765 Рік тому
It's amazing what time and dedication can achieve... with enough Sebastian. ?
@matthias916 Рік тому
Don't underestimate yourself, you too can do these kinds of things enough time and dedication, the dedication is the impressive part
@DynestiGTI Рік тому
Anyone*
@aXYZGaming Рік тому
hello, V
@raditz9676 Рік тому
Whoever you are random stranger on the interwebs, I'm telling you: You can do the same! All you need is dedication, which helps you find the time.
@jeremystott8188 Рік тому
I've always wanted a great demonstration of programming in a similar way someone shows progress of machining something on a lathe/mill/forge/woodworking. Most other videos skip over "all the boring bits" when they get to some coding, but you have absolutely nailed it ❤
@tz4601 Рік тому
Might you be talking about "Stuff Made Here"? It's my only complaint about that channel. I mean, yeah, I'm a programmer so maybe I'm more interested in the programming bits than the average person, and I can also understand it can be difficult to coherently show code in an edited video. But a lot of his projects are something like 50% or more code, and he's a wicked smart programmer, so I wish he included more of it. On the extreme end of "demonstrating programming," Handmade Hero is a (very) long running series of originally-streamed videos where a professional systems game dev builds a modern, professional game/engine from scratch in C. (Really it's C++ but it's a very C-like subset of features.) Even if you don't watch the whole thing, the first ~20-30 videos are enormously enlightening w/r/t the fundamentals of lower level application programming. (e.g., I now know how you can use the Win32 API to open windows, handle keyboard/controller input, play sound, and so on.) Near the end of that first streak of Windows-specific videos he shows off some truly remarkable techniques for custom hot reloading and input recording/playback.
@myavkat5954 Рік тому
@@tz4601 661 videos with each video nearly 1 hour starting from 7 years ago. Just respect. I had the intention to build a game engine with C but then I realized it is much harder to write in pure C and I didnt wanna learn C++. I also realized that I was just motivated by all the youtube videos where people code their games/engines from scratch and had little idea how one works. So I dropped the engine idea for now (planning for later) and started to build a terminal text file editor with C. Even that simple project had gotten me so far (far in my own terms) that I had become comfortable with libraries, compilers, linking them, how to use them searching for old documentations etc. I dont know where I am going with explaining these lol. As for the recommendation you gave, thanks. I probably wont watch it because it is hella long and I like to learn hands on with trial and error, searching for problems on my own. But if I ever start to code an engine and I have a problem that I cant solve no matter what this will probably be my go to source.
@wafje Рік тому
The series of 3Blue1Brown on neural networks is kinda similar and dives even more deeper into the calculus.
@Kibito80 Рік тому
It’s such a bittersweet moment watching your videos as they’ve recently come out because I know such great content with such level of detail takes so long to produce and it’s going to be a long and sad time until your next video comes out. I just love your videos man, everything you touch becomes gold, you make so many topics that are so boringly taught at uni seem sooooo exciting!
@gsp_admirador Рік тому
best video on machine learning that I have ever seen, how is this even free? This is what the internet & youtube was made for.
@YMandarin Рік тому
I love the visual explanations and the hand setting of the weights, its a really intuitive explanation for the networks
@randy7894 Рік тому
Sebastian :"Just open your favorite code editor, type in a few lines and there is your multiverse simulator with special effects" Always a treat to watch your "magic" man :)
@sciencemathematics 11 місяців тому
From just the first couple of minutes of your video, I was able to code my own working classifier. I love you build up everything from first principles and also show your first "naive" implementations before moving on to the more optimized versions. It really makes all the moving parts easy to understand. Excellent work!
@balazskustos9166 Рік тому
BTW, I think even the final challenge is "kind of realistic". Imagine if you were a newborn baby, who never really saw anything in this world, and doesn't know what things are, how things work. And imagine that you have a blurry vision like these low quality images. Also, you can only see static images, without any sound or context. I think this neural network makes a lot better job than any human being could ever do with the same information. Only it does it's learning/thinking a lot faster, so after a few seconds it's not really a toddler anymore, but you got the point...
@janjjd Рік тому
You´re such a gifted educator, I really appreciate the quality and the amount of work that goes into these videos, inspiring stuff!
@odomobo Рік тому
You're such a great educator/communicator. You explained gradient descent in a way that makes simple intuitive sense to me in about 2 minutes, whereas it's something I've only understood abstractly in over 15 years of trying to understand this stuff!
@mdk03 3 місяці тому
When a video on neural networks has the best derivative explanation on youtube
@artiomboyko Рік тому
Calm voice, pleasing visuals, fascinating topic and intuitive explanations. What else would you need... I have actually just created my private playlist for videos I don't want to lose, with this video being the first one added there. Thank you so much!
@stacklysm Рік тому
I couldn't use any other word besides perfect to describe this video. The visuals, explanations and overall progression were exceptional and on point, every concept contributed to the next one, nothing was rushed or under/overly exposed. I wish more people have the opportunity to find your amazing channel, because these videos are truly special.
@proka1 Рік тому
New to the channel, I see :) This guy is great.
@ganymede242 Рік тому
I'm in awe of how good this content is. Production values are superb: voice is really easy to listen to with clear diction and pleasing accent. Graphics clear, smooth, and helpful. Content is out of this world. The explanations are great. I've not seen someone cover the whole thing virtually from scratch and yet at no stage does it feel like we're getting bogged down.
@sibonyhugo5887 Рік тому
Did I just watch an hour long calculus course and not only i think I understood everything and going to try on my own but wanted it to last 2 hours longer? My god this channel never fail to impress me by its quality.
@RichardFredriksson Рік тому
This is a wonderful trip down memory lane; having learnt about this stuff at University. Your ability to visualize everything is amazing, thank you for making such lovely content.
@tagcopperlight Рік тому
Please continue to make hour long videos, it's so relaxing to watch !
@Humberd01 Рік тому
I wish I had seen this video when I was in an AI class back then. It would have been so much easier.
@ZaItan1 Рік тому
I love these intuitive explanations, and appreciate the live demo testing. Drawing pictures of things distinctively *not* one of the labels might help identify issues to keep improving.
@sammyboy1112 Рік тому
One of the best tutorials I have come across for ANNs in the 2-3 years I've been working on them. Very intuitive, bravo. You must really know your stuff.
@intangiblematter_misc Рік тому
Sebastian Lague is the only UKpostsr who can make a 55 minute long video feel like no time at all
@gregor-alic Рік тому
This is actually an amazing explanation of backpropagation, good job! I would recommend to anyone who wants to learn backpropagation to just take a piece of paper and a pencil, and derive those formulas themselves. Trust me, it helped me immensly when I tried to learn backpropagation a few months ago
@AmirGTR Рік тому
The racing line the car was taking was amazing. Perfect apexes.
@charlieahill Рік тому
The caliber, quality and attention to detail in your videos is outstanding. I gained a better, and more fundamental, understanding about neural networks from watching this video that studying them for a whole semester at university. Thank you!
@conradrobinson7941 Рік тому
You have never made a bad video. My favourite video in the notification box updates every time you upload, from your last video to this one. Anyway, point being, you make my favourite videos on this platform.
@rodneydlouhy7409 Рік тому
This Channel has been like Bob Ross for me. I may have very little idea how to code like him, but I love watching how excited he gets over his projects.
@experiment8230 Рік тому
Truly a gifted educator, you had me captivated for an hour on a topic I already knew. Well well done
@loganb2198 Рік тому
This is the best explanation for neural nets I have ever seen. I have watched tons of videos on the subject and never learned anything, but after watching this video I now feel like I have a decent grasp on how they work. I am now inspired to go and try to make one myself. Thank you for your hard work Sebastian, your videos are amazing!
@philurname6575 Рік тому
I've spent many hours learning about ML algorithms, with a lot of that time spent on MLPs. I've working with PyTorch and TensorFlow before. I thought I understood everything pretty thoroughly but just that first example of manually tweaking weights and biases and seeing how those affect the output graph showed me something new. You're videos are incredible and inspire a whole generation of programmers.
@ryanchowdhary965 Рік тому
And then there's me whos stuck on regressions for the past few days.
@NamePointer Рік тому
This is by far the best explanation I have ever seen of Neural networks, not because it is particularly precise, but because it does a fantastic job at making the concepts of nodes, weights, biases, activation functions, gradient descent seem very intuitive, which is rare for such a theoretical subject. Outstanding job!
@ACorgi 6 місяців тому
This is a really neat way of teaching this. I've already studied this stuff some, but I just love this approach. The way you're teaching this from the ground up, showing how to make little enhancements and the rationality behind them with each step. It's so cool.
@Sam_mcl Рік тому
This is one of my favourite videos on UKposts ever. Incredibly well-made and useful! Cheers, Sebastian.
@hanzazazel412 Рік тому
I never understood maths until I started attempting basic code. These videos are the first ones that actually make sense because I can properly visualize the maths.
@skillplants Рік тому
I remember the poll you made about making longer videos. I really appreciate you listening to the fans, I absolutely love these longer videos. Keep it up!
@bradye2588 Рік тому
Your videos are just like air shows. I never quite know what is happening, and almost everything goes over my head, but I still have a lot of fun.
@aiden_3c Рік тому
I really really like how you start from a really simple example and start working up from that, it really makes a neural network a lot easier to understand
@glorytoarstotzka2380 Рік тому
I've been working on a basic neural network just because I wanted to learn this, I already knew the basics but it' so cool to see you working on these. This really helped cement the knowledge I learned.
@alexweinberger8925 Рік тому
Props to you for not just using sklearn or PyTorch! You actually built the NN from nothing. That’s a dream of mine.
@tsuaeghakihas469 Рік тому
I cant tell you how excited when i see one of your videos get uploaded. Somehow i missed this one, THANK YOU so much
@velocitygames9404 Рік тому
This is seriously one of the best videos on neural networks I have ever seen. The amount of work going into these videos is incredible and inspiring! Thank you so much!
@Monkeylordz88 Рік тому
Convolutional NNs are the key to creating good image-recognition networks (at least at the moment). It's pretty amazing how a couple of straightforward image transformations add so much information and allows CNNs to gain such an edge over traditional feed-forward networks. In fact, these convolution-like processes are even used by humans, such as line detection. Your videos and explanations are incredible, I would love to see you dive deeper into the amazing field of ML!
@Flobyby Рік тому
I wonder if a simple luminance/chrominance encoding of images could already significantly improve the performance of a simple networks such as this one for the last exercise.
@ruroruro Рік тому
@@Flobyby unlikely. Maybe, for really small networks it might, but in general feature engineering (manually preprocessing the data) is almost always useless for neural networks, because the network can learn the best transformations for each particular task on its own. Actually, that's kind of the whole point of deep learning - to avoid having to manually hard code the features.
@heyreefes Рік тому
Been learning a lot from you last two weeks. You are amazing
@uncertawn Рік тому
Easily the best vid on neural network. I have been looking for a good explanation on haw neural networks work, but none of them explained it in such understandable way. THANK YOU SO MUCH
@fascher_ Рік тому
I feel like I've learnt a lot, this must have taken ages to make!! I'm in awe of how you're able to visualise complex information, and explain it in simple language. Your videos are also super calming to watch. I think when it comes to maths, or learning new things in general, sometimes people can get a bit stuck or frustrated. So, a video that just states the information plainly can become a stress when someone is learning a new thing. I think your videos are amazing at both explaining the information and delivering it in a calming way that makes people receptive and curious to learn more. It's a skill and you do it very well, so thanks for sharing your passions!
@ET-yc4wb Рік тому
I know you're not gonna see this message, but I hope you know that what you do is amazing. I am still in highschool, haven't learned calculus yet, and even so, your explanation of calculus just made so much sense to me. Everytime I tried to learn calculus I could not wrap my head around what derivatives were or what it is meant to do, but I understood it in a few minutes just from your example! Thank you, Sebastian.
@dantelaviero7782 Рік тому
i've had trouble for 5 years learning derivatives, dropped out of universities cause no one would explain to me why we use them, just how. Your channel is a blessing
@polygontower Рік тому
Did you take the optional math course? It was probably taught there
@tams805 Рік тому
@@polygontower Did you go to the same educational institute as them? What's that? You don't know? Then who are you to make such a comment?
@polygontower Рік тому
@@tams805 No. An educated guess. An educated guess. An educated guess.
@ryanchowdhary965 Рік тому
@@polygontower someone has learned something from grade 11 chapter 1 mathematics. Always take an educated guess.
@RichConnerGMN Рік тому
@@polygontower i just want to know why you repeated that 3 times
@knicklichtjedi Рік тому
One of the best videos for explaining neural networks internal mechanisms without presenting them overly complex. Well done!
@Tommus1997 Рік тому
That's a really nice way of introducing the neural networks. Explaining the different parameters and fiddling with them by hand + the visualization, before starting to explain how we can make our computer do the fiddling. Very cool idea!
@superkaboose1066 Рік тому
How people are this smart just boggles me, I love how well you simplify things though, the little simulations are just perfect once again :)
@zicada7661 Рік тому
Love this long format! Excellent job!
@evanduffy1015 Рік тому
This is far and away the best explanation of NN I've ever seen, and I took college courses on NN's. The included explanation of the calculus behind it was amazing as well
@BjarneSvanberg Рік тому
Lovely video as always! In your doodle of the helicopter problem, you might be testing it with a thinner line (or less opaque) than the training set. Thus if the intencity of the line is affecting the activation of the neurons, you might not get the correct result. This could be the case for the numbers as well. The training set with numbers seems to be drawn with a thicker line than the one you use to test it with.
@pawepiat6170 Рік тому
So, in theory, vectorization of the input would help?
@vrcookingsimulator5458 Рік тому
Hey man I think you would really find plate tectonics simulations interesting. It kind of fits well with your procedural terrain generation series and is extremely interesting to read about.
@stephenwestland942 Місяць тому
This is stunningly good. I love the visualisations.
@noahleon6549 Рік тому
i was riveted and following along where i could, but this was also an amazing ASMR for a nap when i couldn't follow along anymore! wonderful.
@ralph_d_youtuber8298 Рік тому
This is exactly what I learned in my first year calc class. Without the fancy visualization. I feel like I understand it better now 😂
@h4kku Рік тому
I like that you condensed a whole semester course of artificial intelligence at university into 55 minutes :D
@andrewkicha1628 Рік тому
Just amazing visualizations that help to answer the most fundamental questions that popped up in my head. Thank you for this amazing video! I look forward to seeing more content like this on the channel.
@mattjpoolr 4 місяці тому
did not think I'd stay the whole hour, but was fully engaged the whole way through
@realryleu Рік тому
That graph renderer you made looks almost exactly like desmos with a dark theme enforces by the dark reader extension.
@randomizer495 Рік тому
The way he slowly breaks the problems into smaller chunks was making even my small brain could digest it tho:D
@ShikaIE Рік тому
omg this has got to be the best visual explanation of neural network. I have never seen anyone "touching" on how the training happen, it's like magic, but seeing how you have sliders to manually create boundaries is really mind-blowing!
@isaacgreen9495 Рік тому
This is absolutely incredible. I wish I had had this video when I was learning to build neural networks. Definitely the best I’ve seen by far. Your visuals are incredible!
@polimetakrylanmetylu2483 Рік тому
It's a great video and I really appreciate the effort you put into visualizing those concepts :) I made a little note 6:50, you said that it doesn't make sense to change the size of input or output. For neural networks it would be impractical, yeah. However, for many other classifiers you can increase the number of inputs, that includes making logistic model to have a "bendy" decision boundary. You may for example add an input and make it be a nonlinear function of another input, like input one squared, sine of input two, euclidean distance between a data point and 0,0. Then you can train linear model on this augmented data and it will be able to have bendy decision boundary in input space. That's what eg. SVMs use and it's called a kernel trick - making a nonlinear problem a linear problem in nonlinear space
@SebastianLague Рік тому
Good point, thank you!
@WaluigiisthekingASmith Рік тому
The learn rate part reminded me of damped harmonic oscillation. For a damping coefficient that's too low it will start to oscillate around the steady state, if it's too high it will decrease but not very fast, but if it's critically damped it will return as fast as possible to the bottom.
@samuelwerley528 Рік тому
Yes these are very similar concepts
@RockuHD Рік тому
This video is fantastic! Most videos and tutorials don't explain it as well or as in much detail. You can really see how it started so simple and how it got to the end result! Thank you for clearing up this subject for me! I will try and write one based off the subjects in this video (not ripping the code ofc)! thank you
@AdrianHereToHelp Рік тому
This video is legitimately incredible. I genuinely believe this is one of the best explanation videos I've seen, across this whole site.
@Dysiode Рік тому
23:22 -ish I think it might be helpful to note that the derivative describes the rate of change (i.e. acceleration) of a function at every point, whereas the delta gives the rate of change at a single point. I'm at 31:37 and I had just remember that bit and found it interesting.
@colinbrown7947 Рік тому
Acceleration would actually be the rate of change of the rate of change, whereas velocity is the rate of change. And to be more mathematically precise, you're talking about the difference between the derivative at a point of a function vs the derivative of a function. A delta is always an approximation based on some small nudge.
@neelbanga Рік тому
Hey! I'm a huge fan, your videos have inspired the way I create content, my latest video takes a lot of inspiration from your style of content. I just wanted to say thanks for both your inspiration and great content!
@lordtrollalot8707 Рік тому
the genius can explain most complex prolems in a easy way. here is the best explaination how neural networks work, ive ever seen.
@emilien237 Рік тому
This video is probably the best video about neural network and machine learning I have ever seen. Everything from the calculus to the neural network itself was really well explained.
@dcode1 Рік тому
That feeling when you spent months creating an AI that can drive a car (here is the video ukposts.info/have/v-deo/hadnfa1_ZKxnso0.html ) and then Sebastian comes in and just casually drops in the intro ( at around 1:01 ) that he wanted to do "something simpler" and shows his version of an AI that can drive a car 😅
@ashpun5998 Рік тому
Awesome.(I haven't watched it yet though.)
@ashpun5998 Рік тому
And if it was uploaded a few seconds ago, how are there comments from 15+ hours before?
@Extex_ Рік тому
Was marked as private and he shared the link with some people before releasing it. Probably his Patreon
@Prodrozer11 Рік тому
@@ashpun5998 he probaly posted it already and set it on private after, probaly to edit corrections and stuff etc
@oriyadid Рік тому
@@Prodrozer11 don't think so, his Patreon members get early access to videos
@ashpun5998 Рік тому
Thanks guys.
@bernardmcdonald5257 Рік тому
I don’t often comment on UKposts videos, but I just had to say that you are one of my favorite UKpostsrs. Your content is amazing, and you explain/visualise the concepts and code in such a compelling and entertaining way.
@KeygaLP Рік тому
This is BY FAR the best introduction to neural networks I've ever seen. The visualizations really helped. Thanks for the great content!
@user-sv3fz3pg4z 2 місяці тому
Many thanks for this great video! It helped me a lot understand how a neural network works internally and even better how to program it
Суспільне Львів
Переглядів 421 тис.
Layerwise Lectures
Переглядів 638 тис.
Samson Zhang
Переглядів 1,8 млн