But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning
Переглядів 2,163,802
День томуUnpacking how large language models work under the hood
Early view of the next chapter for patrons: 3b1b.co/early-attention
Special thanks to these supporters: 3b1b.co/lessons/gpt#thanks
To contribute edits to the subtitles, visit translate.3blue1brown.com/
Other recommended resources on the topic.
Richard Turner's introduction is one of the best starting places:
arxiv.org/pdf/2304.10557.pdf
Coding a GPT with Andrej Karpathy
• Let's build GPT: from ...
Introduction to self-attention by John Hewitt
web.stanford.edu/class/cs224n...
History of language models by Brit Cruise:
• ChatGPT: 30 Year Histo...
Paper about examples like the “woman - man” one presented here:
arxiv.org/pdf/1301.3781.pdf
------------------
Timestamps
0:00 - Predict, sample, repeat
3:03 - Inside a transformer
6:36 - Chapter layout
7:20 - The premise of Deep Learning
12:27 - Word embeddings
18:25 - Embeddings beyond words
20:22 - Unembedding
22:22 - Softmax with temperature
26:03 - Up next
------------------
These animations are largely made using a custom Python library, manim. See the FAQ comments here:
3b1b.co/faq#manim
github.com/3b1b/manim
github.com/ManimCommunity/manim/
All code for specific videos is visible here:
github.com/3b1b/videos/
The music is by Vincent Rubinetti.
www.vincentrubinetti.com
vincerubinetti.bandcamp.com/a...
open.spotify.com/album/1dVyjw...
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on UKposts or otherwise following on whichever platform below you check most regularly.
Mailing list: 3blue1brown.substack.com
Twitter: / 3blue1brown
Instagram: / 3blue1brown
Reddit: / 3blue1brown
Facebook: / 3blue1brown
Patreon: / 3blue1brown
Website: www.3blue1brown.com
@3blue1brown Місяць тому
Edit: The finalized version of the next chapter is out ukposts.info/have/v-deo/nX2drGudfYSgupc.html Early feedback on video drafts is always very important to me. Channel supporters always get a view of new videos before their release to help inform final revisions. Join at 3b1b.co/support if you’d like to be part of that early viewing group.
@bbrother92 Місяць тому
@3Blue1Brown thanks explaining these things - it is very hard for web programmer to undestand math
@JohnSegrave Місяць тому
Grant, this is so good! I've worked in ML about 8 years and this is one of the best descriptions I've seen. Very nicely done. Big 👍👍👍
@deker0954 Місяць тому
Is this worth understanding?
@bbrother92 Місяць тому
@@JohnSegrave sir, could you recomend video analysis framework any video description model?
@didemyldz1317 Місяць тому
Could you share the name of the model that is used for text-to-speech generation ? Me and my teammate are working on a Song Translator as a senior design project. This might be very helpful. Thanks in advance :)
@DynestiGTI Місяць тому
Grant casually uploading the best video on Transformers on UKposts
@drgetwrekt869 Місяць тому
i was expecting froggin electromagnets to be honest :-)
@brandonmoore644 29 днів тому
This video was insanely good!
@shoam2103 28 днів тому
Even having a basic understanding of what it is, this was still extremely helpful!
@yigitpolat 28 днів тому
yeah but it did not talk about transformers in this chapter
@stefchristensen47 27 днів тому
I wish I could retweet this post.
@iau Місяць тому
I graduated from Computer Science in 2017. Back then, the cutting edge of ML were Recurrent Neural Networks, in which I based my thesis. This video (and I'm sure the rest of this series) just allowed me to catch up to years of advancements in so little time. I cannot describe how important your teaching style is to the world. I've been reading articles, blogs, papers on embeddings and these topics for years now and I never got it quite like I got it today. In less than 30 minutes. Imagine a world in which every teacher taught like you. We would save millions and millions of man hours every hour. You truly have something special with this channel and I can only wish more people started imitating you with the same level of quality and care. If only this became the standard. You'd deserve a Noble Prize for propelling the next thoustand Nobel Prizes.
@lucascorreaaa Місяць тому
Second that!
@kyo250996 Місяць тому
Same, I did a thesis about vectorize word back in 2017 and no one ever talked about the whole vector of word gives rise to meaning and context when you generate phrases. Too bad since noone was interested in ML back then, I leaned on web development and drop the ML :(
@iankrasnow5383 Місяць тому
Funny enough, the other 6 videos in this series all came out in 2017, so you probably didn't miss much.
@XMysticHerox Місяць тому
Well transformers were first developed in 2017 so it was the cutting edge exactly when you graduated ^^
@rock_sheep4241 Місяць тому
This is explained in layman terms, but in reality is more complicated than this
@tempo511 28 днів тому
The fact that meaning behind tokens is embedded into this 12000 dimensional space, and you get relationships in terms of coordinates and direction, that exists across topics is mind blowing. Like, Japan -> sushi is similar to Germany -> bratwurst is just so darn neat
@nctbeats7091 28 днів тому
I actually went berserk when I saw that part of the video, so friggin cool.
@amarissimus29 28 днів тому
And it makes the absurdly ham fisted model tampering behind debacles like the Gemini launch look even more absurd. I can hear the troglodytes mobbing in the nth dimension.
@dayelu2679 28 днів тому
I‘ve come to this realization long time ago then I want to find isomorphic structures of concepts across different disciplines
@TheKoekiemonster1234 27 днів тому
@@dayelu2679🤓
@stefchristensen47 27 днів тому
You can actually try this out in your nearest large language model, like ChatGPT, CoPilot, Gemini, or Mistral. Just ask it to do vector math on the words. Since there isn't a predefined vector word calculus in English, the LLM defaults to just using a version of its own internal representation, and so it can eke out pretty results. I was able to duplicate Hitler - Germany + Italy = Mussolini and sushi - Japan + Germany = sausage (or bratwurst, bother score highly) in GPT-3.5-Turbo Complete. It also figured out sushi - Japan + Lebanon = shawarma; sushi - Japan + Korea = kimchi; Hitler - Germany + Spain = Franco; and Hitler - Germany + Russia = Stalin.
@keesdekarper Місяць тому
This video is gonna blow up. The visualizations will help many people that aren't familiar with NN's or Deep Learning to at least grasp a little bit what is happening under the hood. And with the crazy popularity of LLM's nowadays, this will for sure interest a lot of people
@TheScarvig 27 днів тому
as someone who gave a lot of fellow students lessons in stem field classes i can tell you that the sheer amount of numbers arranged in matrices will immediately shut down the average persons brain...
@lesselp 24 дні тому
No, normal people just want to party.
@billbill1235 Місяць тому
I was trying to understand chatGPT through videos and texts on the Internet. I always said: I wish 3b1b releases a video about it, it's the only way for someone inexperienced to understand, and here it is. Thank you very much for your contributions to youtube!!
@lmao8207 Місяць тому
no even the other videos are kinda meh, even if youre not inexperienced because they dont go in depth, i feel here people get a nice understanding of the concepts captured by the models instead of just the architecture of the models
@goldeer7129 Місяць тому
It's kind of true, but if I had to recommend a good place to actually understand transformers and even other machine learning things I would definitely recommend StatQuest, its levels of clearly explaining what's going on are very high. But I'm also very excited to see how 3B1B is going to render all that visually as always
@himalayo Місяць тому
I was also just looking into transformers due to their extreme takeover in computer vision!
@baconheadhair6938 Місяць тому
shoulda just asked chatgpt
@ironmancloud9759 Місяць тому
NLP specialization by Andrew covered everything 😅
@Silent_Knife Місяць тому
The return of the legend! This series is continuing, that is the best surprise of UKposts, thanks Grant, you have no idea how much the young population of academia is indebted to you.
@kikiroy5178 Місяць тому
I'm 26, young engineer. Thinking the same. Well said.
@youonlytubeonce 29 днів тому
I liked your comment because I'm sure you're right but don't be ageist! 😊 Us olds love him too!
@samad.chouihat4222 28 днів тому
Young and seniors alike
@lewebusl 29 днів тому
This is heaven for visual learners. Animations are correlated smoothly with the intended learning point ...
@gorgolyt 24 дні тому
There's no such thing as visual learners. Other than the blind, all humans are visual creatures. It's heaven for anyone who wants to learn.
@lewebusl 24 дні тому
@@gorgolyt You are right. The human get input from 5 senses , but 90 percent of the brain receptors are directly connected to optical and auditory nerves. That is where the visual dominates the other senses ... For blind people the auditory dominates...
@Kargalagan 29 днів тому
I wish i had a friend as passionate as this channel is. It's like finding my family I've always wanted to have
@katech6020 27 днів тому
I wish the same thing
@user-vt4bz2vl6j 26 днів тому
become friends already you both
@TheXuism 20 днів тому
here we are 3b1bro now
@cagataydemirbas7259 19 днів тому
Lets become friends
@parenchyma Місяць тому
I don't even know how many times I'm going to rewatch this.
@chlodnia 29 днів тому
True
@armanahmadian4373 29 днів тому
3B1B doens't need to be saved in watch later folder because all his videos are worth watching later.
@synthclub 29 днів тому
What will you set your weights n biases too?
@oofsper 29 днів тому
same
@chase_like_the_bank Місяць тому
You *must* turn the linguistic vector math bit into a short. -Japan+sushi+germany=bratwurst is pure gold.
@XMysticHerox Місяць тому
I am slightly offended it did not result in "Fischbrötchen".
@marshmellominiapple 28 днів тому
@@XMysticHerox It was trained in English words only.
@XMysticHerox 28 днів тому
@@marshmellominiapple ChatGPT supports 95 languages. Not all equally well. But as a German yes it works just as well with german as it does with english.
@-Meric- 27 днів тому
@@marshmellominiapple Word2Vec and other vector embeddings of words like glove or whatever don't care about language. They don't "understand" the meaning of the words, they just eventually find patterns in unstructured data to create the embeddings. It works in any language and GPT has a ton of other languages in its training data
@stefchristensen47 27 днів тому
You can actually try this out in your nearest large language model, like ChatGPT, CoPilot, Gemini, or Mistral. Just ask it to do vector math on the words. Since there isn't a predefined vector word calculus in English, the LLM defaults to just using a version of its own internal representation, and so it can eke out pretty results. I was able to duplicate Hitler - Germany + Italy = Mussolini and sushi - Japan + Germany = sausage (or bratwurst, bother score highly) in GPT-3.5-Turbo Complete. It also figured out sushi - Japan + Lebanon = shawarma; sushi - Japan + Korea = kimchi; Hitler - Germany + Spain = Franco; and Hitler - Germany + Russia = Stalin.
@y337 10 днів тому
This guy taught me how to build a neural network from scratch, I was waiting for this video, I even posted a request for it in the subreddit for this channel. I’m very glad this finally exists
@shaqtaku Місяць тому
I can't believe Sam Altman has become a billionaire just by multiplying some matrices
@Dr.Schnizzle 22 дні тому
You'd be surprised at how many billionaires got there from multiplying some matrices
@spanishflea634 22 дні тому
Also, gets away with calling it "machine learning".
@tiborsaas 22 дні тому
It's too much reduction, he added value on a higher level. But yeah, when you look deep enough, everything stops looking like magic.
@user-gw3yb3ki6w 22 дні тому
@@tiborsaas And that is a good thing in many cases, it casts away illogical fears when you understand that there is no any kind of magic or thinking behind this. In practice it is just overhpyed guessing machine what word normally might come after X.
@kylev.8248 20 днів тому
@@user-gw3yb3ki6w this concept comes from 2017. We should actually be very very worried and keeping our eye closely on the progress that AI is making. The amount of progress they have made since the 2017 paper 📝 “Attention is all you need “ is insane.
@JustinLe Місяць тому
here's to hoping this is not an April fools
@anuragpranav Місяць тому
it is - you would be a fool to not watch this video
@tinkuefu09 Місяць тому
It's 2nd April here
@TheUnderscore_ Місяць тому
@@anuragpranavEven if you already know the subject? 😂
@me0101001000 Місяць тому
@@TheUnderscore_ it's never a bad idea to review what you know
@anuragpranav Місяць тому
@@TheUnderscore_ you are almost certainly limiting what you might know with that approach
@nicholaitukanov1162 Місяць тому
I have been working on transformers for the past few years and this is the greatest visualization of the underlying computation that I have seen. Your videos never disappoint!!
@brian8507 Місяць тому
So if we "stop" you... then we avoid judgement day? We should meet for coffee
@giacomobarattini1130 Місяць тому
@@brian8507 "judgement day" 😭
@beProsto Місяць тому
@@brian8507 bro's got underlying psychological issues
@talharuzgarakkus7768 Місяць тому
I agree with you . Visualization is perfect way to understanding transformer architecture. Specifically attention mechanism
@jawokenn8766 Місяць тому
@@giacomobarattini1130its later than you thinj
@etiennedud 6 днів тому
The visual of this video are next level, really help the comprehension of the subject
@lucasamadsen 29 днів тому
2 years ago I started studying transformers, backpropagation and the attention mechanism. Your videos were a corner stone for my understanding of those concepts! And now, partially thanks to you, I can say: “yeah, relatively smooth to understand”
@DaxSudo Місяць тому
Writing my first academically published paper on AI rn and I have to say as a engineer in this space, this is one of the most complete and well nuanced explanations of these tools. Gold, nay platinum standard for educational content on this topic for decades to come.
@hyperadapted 28 днів тому
Yes. I really hope that he gets some lifetime achievement massive-footprint-in-a-good-sense type of award in the MINT Edu field.
@1bird_d Місяць тому
I always thought when people in the media say, "NO ONE actually understands how chat GPT works" they were lying, but no one was ever able to explain it in layman's terms regardless. I feel like this video is exactly the kind of digestible info that people need, well done.
@alexloftus8892 Місяць тому
Machine learning engineer here - plenty of people understand how the architecture of chatGPT works on a high level. When people in the media say that, what they mean is that nobody understands the underlying processing that the parameters are using to go from a list of tokens to a probability distribution over possible next tokens.
@kevinscales Місяць тому
It's not a lie, it's just not very precise. No one can tell you exactly why one model decided the next word is "the" while another decided the next word is "a" and in that sense no one understands how a particular model works. The mechanism for how you train and run the model are understood however.
@lolololo-cx4dp Місяць тому
@@kevinscalesyeah just like any deep ANN
@metachirality Місяць тому
Think of it as the difference between knowing how genetics and DNA and replication works vs. knowing why a specific nucleotide in the human genome is adenine rather than guanine. There is an entire field of machine learning research dedicated to understanding how neural nets work beyond the architecture called AI interpretability.
@KBRoller Місяць тому
No one fully understands what the learned parameters mean. Many people understand the process by which they were learned.
@jaafars.mahdawi6911 27 днів тому
Man! You never fail to enlighten, entertain, and inspire us, nor do we get enough of your high-quality, yet very digestible, content! Thank you, Grant!
@xiangzhang5279 27 днів тому
I have always been blown away by how great your visualization is for explaining ML concepts. Thanks a lot!
@yashizuko Місяць тому
Its astonishing, amazing that this kind of info and explaination quality is available for free, this is way better than a University would explain it
@lonnybulldozer8426 Місяць тому
Universities are buildings. Buildings can't talk. Therefore, they cannot explain.
@Mutual_Information Місяць тому
Grant shows just how creative you can get with linear algebra. Who would have guessed language (?!) was within its reach?
@abrokenmailbox Місяць тому
Look up "Word2Vec", it's an interestingly explored idea.
@Jesin00 Місяць тому
Linear algebra would not be enough, but a nonlinear activation function (even one as simple as max(x, 0)) makes it enough to approximate anything you want just by adding more neurons!
Місяць тому
Given words are descriptors and numbers are just arbitrarily precise adjectives... aka descriptions...
@Mutual_Information Місяць тому
@@Jesin00 Yes, lin alg alone isn't enough.
@psychic8872 Місяць тому
Well ML uses linear algebra and he just explains it
@haorancheng4870 24 дні тому
I listened to my professor explaining the crazy equation of softmax for a semester already, and you explained it so well with how temperature also plays a role there. Big RESPECT!
@mariusfurst4898 29 днів тому
Your teaching skills are beyond compare. The effort you put into your videos clearly shows.
@TheMuffinMan Місяць тому
Im a mechanical engineering student, but I code machine learning models for fun. I was telling my girlfriend just last night that your series on dense neural networks is the best to gain an intuitive understanding on the basic architecture of neural networks. You have no idea what a pleasant surprise it was to wake up to this!
@baconheadhair6938 Місяць тому
good man
@keesdekarper Місяць тому
It doesn't have to be just for fun. I was also in Mechanical Engineering, picked a master in control theory. And now I get to use Deep learning and NN's for intelligent control systems. Where you learn a model or a controller by making use of machine learning
@PiercingSight Місяць тому
Straight up the best video on this topic. The idea that the dimensions of the embedding space represent different properties of a token that can be applied across tokens is just SO cool!
@JonnySolomon Місяць тому
i felt that
@MagicGonads Місяць тому
orienting and ordering the space (called the 'latent' space) so that the most significant directions come first is called 'principal component analysis' (useful for giving humans the reigns to some degree since we get to turn those knobs and see something interesting but vaguely predictable happen)
@andrewdunbar828 Місяць тому
I agree. I starting writing about that in a comment about 2 seconds into the video before I knew how well he was going to cover it since it's usually glossed over way too much in other introductions to these topics.
@jerryanyu8467 29 днів тому
Thank you! You're so late 3Blue1Brown, it took me 10 hours of videos + blogs last year to understand what a transformer is! This is the long waited video! I'm sending this to all my friends.
@justchary 15 днів тому
The quality of these videos and depth of openings of the deeper meaning is simply mind blowing
@shubhamz2464 Місяць тому
This series should continue. I thought it was dead after the 4th video. Lots of love and appreciation for your work
@codediporpal Місяць тому
18:45 This the the clearest layman explanation of how attention works that I've ever seen. Amazing.
@tielessin 29 днів тому
It's absolutely ridiculous how many aspects of this topic finally clicked for me in this intro video already. This was incredibly well explained an I'm so thrilled for the next chapters. Thank you very much, Grant!
@CODE7X 28 днів тому
Im in highschool, and i only knew broken pieces of how it works , but you really connected all the pieces together and added the missing ones
@joaoguerreiro9403 Місяць тому
We need more Computer Science education like this! Amazing 🔥
@pythonconsultant Місяць тому
Honestly I hope that in future, AI can produce such great content. This will probably tend to take a couple of years more, but I guess its possible. Even better: You got your own Curriculum based on your strengthens and weaknesses. For me this would be a combination of fireship and 3blue1brown content...
@kalashshah6234 28 днів тому
This is absolutely one of the best videos for explaining the workings of LLMs. Love the visualisation and the innate ease with which the concepts were explained. Hats off!!
@jessylikesithard 27 днів тому
This is by far the most organized explanation i've seen about transformers.
@punkdigerati Місяць тому
I appreciate that you explain tokenization correctly and the usefulness of simplifying it. Many explanations skip all that and just state that the tokens are words.
@pw7225 29 днів тому
Apart from the fact that tokens CAN actually be longer than a word, too. :) Sub-word token does not mean that tokens must be smaller than a word.
@ratvomit874 22 дні тому
There is a related idea here in how Roombas navigate houses. They clearly are forming a map of your house in their memory, but there is no guarantee they see it the same way we do i.e. the different zones they see in your house may not correspond nicely to the actual rooms in the house. In the end, though, it doesn't really matter, as long as the job gets done correctly
@avishshah2186 Місяць тому
You made my day!! This topic was taught at my grad school and I needed some intuition today and you have uploaded the video!!! It seems you heard me!!Thanks a ton!! Please upload video of Vision Transformers, if possible
@karlstanley8264 Місяць тому
This is genuinely one of the best pieces of science / tech communication I have ever seen. Well done and thank you!
@ogginger 21 день тому
You are such an AMAZING teacher. I feel like you've really given thought to the learners perception and are kind enough to take the time and address asides and gotchas while you meticulously build components and piece them together all with a very natural progression that's moving towards "something" (hopefully comprehension). Thank you so much for your time, effort, and the quality of your work.
@owenleynes7086 Місяць тому
this channel is so good at making math interesting, all my friends think im wack for enjoying math videos but its not hard to enjoy when you make them like this
@SidharthSisawesome Місяць тому
The idea of describing a vector basis as a long list of questions you need to answer is exactly the teaching tool I needed in my kit!! I love that perspective!
@BlayneOliver 29 днів тому
the 'embeddings beyond words' segment levelled up my understanding of how machine learning 'thinks'. thank you!
@50sKid Місяць тому
Seriously amazing content. The way you present things visually is excellently done.
@Skyace13 Місяць тому
So you’re telling me computer models can quantify “a few” or “some” based on how close the value is to a given word of a number from its usage from training data? I love this
@andrewdunbar828 Місяць тому
Well, a bit.
@XMysticHerox Місяць тому
Well it can encode any semantic meaning only really limited by the number of parameters and quality of training data.
@gpt-jcommentbot4759 29 днів тому
@@XMysticHerox quantity*
@connorgoosen2468 Місяць тому
This couldn't have come at a better time for me! I'm very excited for this continuation of the series. Thanks Grant!
@johncox9099 28 днів тому
Thank you so much for this video. I have watched many on the topic and they teach you some parts, but you connected all the missing dots in a very effective way. Thanks.
@TheVirgile27 29 днів тому
I've been following this high-quality channel for years. And I don't know how it continues to improve over time. Thank you for your hard work of popularization of complex notions and your work of aid to intuition with increible visual representations, really I take my hat off to you, once more : Thank You
@eloyfernandez8668 Місяць тому
The best video explaining the transformer architecture that I've seen so far... and there are really good videos covering this topic. Thank you!!
@Astronomer6573 Місяць тому
Your explanation tends to always be the best! Love how you visualise all these.
@SenseiCC 27 днів тому
This channel is so good!! The way such complicated topics are broken down and explained is really of the highest standard. Please never stop making videos!
@junjalapeno7773 28 днів тому
As someone working in IT, I can imagine how much blood, sweat and tears were involved in coding and testing this, the number of meetings and arguments with the product owner, project manager and the management to come up with this complex and beautiful infrastructure. We can't even deploy a fucking CRM in peace
@RyNiuu Місяць тому
ok, you read my mind. From all of the channels, I am so glad, it's you explaining Transformers.
@MaxGuides Місяць тому
Amazing work, your simple explanations in other videos in this series really helped me get a better understanding of what my masters classes were covering. Glad to see you’re continuing this series! ❤
@Noriyak1 28 днів тому
Your videos are the only ones I watch multiple times to get the hang of it. But the way you visualize and explain it makes it way more enjoyable and interesting. Love your videos ❤
@dima13693 29 днів тому
I usually gloss over your videos as it gets more technical. But whatever you did this time, kept me hooked the whole time.
@user-ew1ic7pr3r Місяць тому
I know the material of this chapter very well. Still, I watched it in its entirety just for the pleasure of watching a masterful presentation, the restful and authoritative cadence of the voice, and the gorgeous animation. Well done, Grant, yet again.
@StephaneDesnault Місяць тому
Thank you so much for the immense work and talent that goes into your videos!
@bigmig4356 25 днів тому
Ooh attention ! Looking forward to it. Thank for feeding our attention span with such quality visualisations. From gradients to signifiant coordinates in concept space your visual language keeps getting more refined and synthetic. The adventure through back propagation was rather mind-bending. This series is amazing.
@thelambda5900 Місяць тому
I wish I had you as a teacher. You make math so much more fun than I know it already❤
@ranajakub Місяць тому
this is the best series from you by far. excited for its revival
@jorgeromeu Місяць тому
Hi Grant, when I first saw your original 5 deep learning videos I was in my second to last year of high-school, they were my first introduction to ML and deep learning, and they played a part in me choosing to study Computer Science as an undergraduate. Now, five years later I am working on my masters thesis where I am using Vision Transformers :)
@pingsunday 13 днів тому
the amount of work in this video is huge like the mountain of Fuji
@ahmedivy Місяць тому
Without watching i can say that this is going to be the best transformers video on yt
@robertwiebe Місяць тому
Right you are.
@Musthafamum 28 днів тому
It is
@scolton Місяць тому
Most exciting part of my week by far
@ayushranjan2494 24 дні тому
The amount of depth and efforts invested in this video, makes this channel the best channel on UKposts.
@asdads3948 Місяць тому
Love that bit about the word embeddings and how each direction in that high dimensional space carries some semantic meaning to a certain degree. Haven't heard something i found that interesting in years!
@jortand Місяць тому
Damit nice April fools joke, I got fooled into learning something.
@zmaron1 Місяць тому
The BEST AI video. Highly recommended !
@burakbasci Місяць тому
A few month ago I searched for explanation for this topic, but all I found was either way to complex or overly simplified. I really admire how you manage to bridge that gap!
@kunalpant3886 Місяць тому
Being an aspirational ML engineer, this whole series is the best explanation on Neural Networks, Transformers and overall Deep learning that I've come across all over the web
@vikrambhutani 29 днів тому
I love 3Blue1Brown series ..the linear algebra series was really the state-of-the-art and recognized globally for AI enthusiasts like myself. Now hot topics such as Transformers and GenAI , this is really the best explanation by far. Its short and precise and that's what we want.
@viola_case Місяць тому
Deep learning is back baby!
@kevinscales Місяць тому
A short 6 year 5 month wait!
@Hablo74 29 днів тому
Wow!! I've no prep in math, analisis, programmig, machie learning, etc... and can grab the concept! What a MIRACLE have you done here! Good work! 🤯
@eriktruong9856 Місяць тому
Thank you so much. We were just learning about transformer architecture in our deep learning course. This was really helpful for visualizing word embeddings!
@bridgeon7502 Місяць тому
Hang on, I thought this series was done! I'm delighted!
@tomasretamalvenegas9294 18 днів тому
CHILE MENTIONED 🇨🇱🇨🇱❤️❤️🇨🇱🇨🇱🇨🇱 COME TO SANTIAGO GRANT!!!
@shreyasjena474 29 днів тому
I'm currently working with Transformer models on a regular basis, and still your video helped me look at underlying ideas in a new way. Thanks as always Grant!
@user-cw7ei9yj6b 27 днів тому
This is, with no exaggeration, the best video series I have ever seen diving deep into the architecture of transformers and large language models. Thank you infinitely.
@davidm2.johnston684 Місяць тому
Hello 3b1b, I wanted to say a huge thank you for this specific video. This was exactly what I've been needing. Every now and again, I thought to myself, as someone who's been interested in machine learning for my whole adult life, that I should really get a deep understanding of how a transformer works, to the point that I could implement a functional, albeit not efficient, one myself. Well, I'm on my way to that, this is at least a great introduction (and knowing your channel I really mean GREAT), and I really wanted to thank you for that! I know this is not much, but I'm not in a position to support this channel in a more meaningful way at the moment. Anyways, take care, and thanks again!
@3blue1brown Місяць тому
I'm glad you enjoyed. In case some how you haven't already come across them, I'd recommend the videos Andrej Karpathy does on coding up a GPT. In general, anything he makes is gold.
@actualBIAS Місяць тому
OH MY GOODNESS Your timing is just right! I'm learning about deep neural nets and transformers will be my next topic this week. I'M SO EXCITED, I JUST CAN'T HIDE IT! I'M ABOUT TO LOSE MY MIND AND I THINK I LIKE IT!
@looppp 24 дні тому
The word embedding difference example is.. incredible I never thought about it this way Thank you so much for this!
@55114706 Місяць тому
this video is amazing!! finally after all these years, i asked many "Data Scientist" about Network Propagations and Deep Learning Neural Networks, many of them responded it's a "black-box" algorithm. your video here really uncovers the black-box.. i am very amazed by the sheer quality of your video. this is an amazing masterpiece!
@Jackson_Zheng Місяць тому
YOU DID IT! I emailed you about this video idea about 8 months ago and I've been patiently waiting for you to release this since!
@user-vb8lx8pi6o Місяць тому
wow, great idea!
@melihozcan8676 29 днів тому
YOU DID IT JACKSON! I texted you to email him this idea about 9 months ago. Now the bab- video is there!
@gONSOTE 29 днів тому
SANTIAGO DE CHILE MENTIONED!!! 🗣️🔥🔥🔥 WHAT THE HELL IS CLEAN AIR!!?!?!?!? 🗣️🗣️🗣️🔥🔥
@giacomofumagalli8532 Місяць тому
Gold resource! Great communication skill! Always a pleasure to listen to. Thank you
@rolinejohnaguilar5272 26 днів тому
It amazing that this knowledge is free, really learned a lot from this short session. Definitely will binge watch your videos.
@BobbyL2k Місяць тому
As an ML researcher this is an amazing video ❤. But please allow me to nitpick a little at 21:45 It’s important to note that while the “un-embedding layer” of a Transformer typically have a different set of weights from the embedding layer, in OpenAI’s GPT model each vector for each word in the un-embedding layer is exactly the same vector as ones in the embedding layer. This is not the case for Transformer models that has the output be in a different domain than the input (e.g, translating to a different language), but since the video is specifically talking about GPT. This is the specific of the implementation detailed in the “Improving Language Understanding by Generative Pre-Training” paper by OpenAI. The reusing weights make sense here because each the vector from the embedding is a sort of “context free” representation of the word. So there is not need to learn another set of weights.
@dhruvshah3909 Місяць тому
I started my deep learning journey from your original videos on deep learning. They inspired me to work in this field. I am about to start my first internship as a researcher in this field. Thank you 3blue1brown for this.
@dhruvshah3909 Місяць тому
Also this is the best video that I have seen through my many hundred videos from when I was stuck in tutorial hell on many of these concepts
Місяць тому
Just in time to be replaced by them >:).
@zoebluberry1315 Місяць тому
And again and again and again in the pool of Million explenations, Yours is the one that makes the topic understandable for everyone who is able to watch and listen. Great work!
@garlapatiraviteja2104 25 днів тому
I initially learned about transformers through text, but watching this video with its awesome gifs really brought the concept to life! Such a visually captivating way to understand complex topics.
@kalin4452 Місяць тому
Before I clicked on this video, I thought a Transformer was a fictional machine-like species that was being used as toys. Now I know that transformers are much more different. Thanks Grant.
@trevinbeattie4888 Місяць тому
They’re more than meets the eye. ;)
@z-beeblebrox Місяць тому
3blue1brown released a normal video today. So did Numberphile. So did nearly all the channels in my subsd. There's no wacky bullshit on the google homepage. No stupid gimmick feature in Maps. Have we done it? Have we finally killed off the lamest holiday? Is it finally dead?
@ahmedsaleh5141 3 дні тому
You are the greatest contributor to human knowledge on UKposts. Thank you.
@mklcst Місяць тому
This pure gold. Amazing. The quality of your videos is the highest on YT.
@minds_and_molecules Місяць тому
The different sampling has to do with the search algorithm, like beam search, or any search involving topk or some tally of probabilities for the final score of the output. Any temperature will not change that the most probable token is the most probable token, so in a greedy search the temperature does not affect the output. This is a very common misconception, I'm a bit disappointed that it was slightly misleading here.
@alfredwindslow1894 Місяць тому
agree, what he said wasn’t logically complete and didn’t really make sense because of it
@minds_and_molecules Місяць тому
To be clear, rest of the video was great!
@henryrugg4971 Місяць тому
I'm a simple man. I see 3B1B has released a new video, I click...
@playmaker2404 29 днів тому
i was searching for next part but later saw this is a recent video, next video would be tough as complex self-attention is involved hoping to learn it in best possible way :D
@teegeevee42 Місяць тому
Thank you for explaining this so clearly. I'm looking forward to the next videos of the series.
@user-qn2bg7zb9s Місяць тому
So sad that so many people think AI picks bits of text and images directly from data and just makes a collage...
3Blue1Brown
Переглядів 831 тис.
Р@ССИЯН ПРИВОЗЯТ ПО ЧАСТЯМ! Он пойдет на войну, умрёт, а я куплю квартиру и оденусь/Золкин, Карпенко
Апостроф TV
Переглядів 1,7 млн
RationalAnswer | Павел Комаровский
Переглядів 1,1 млн