Transformers, explained: Understand the model behind GPT, BERT, and T5

  Переглядів 863,075

Google Cloud Tech

Google Cloud Tech

День тому

Dale’s Blog → goo.gle/3xOeWoK
Classify text with BERT → goo.gle/3AUB431
Over the past five years, Transformers, a neural network architecture, have completely transformed state-of-the-art natural language processing. Want to translate text with machine learning? Curious how an ML model could write a poem or an op ed? Transformers can do it all. In this episode of Making with ML, Dale Markowitz explains what transformers are, how they work, and why they’re so impactful. Watch to learn how you can start using transformers in your app!
Chapters:
0:00 - Intro
0:51 - What are transformers?
3:18 - How do transformers work?
7:41 - How are transformers used?
8:35 - Getting started with transformers
Watch more episodes of Making with Machine Learning → goo.gle/2YysJRY
Subscribe to Google Cloud Tech → goo.gle/GoogleCloudTech
#MakingwithMachineLearning #MakingwithML
product: Cloud - General; fullname: Dale Markowitz; re_ty: Publish;

КОМЕНТАРІ: 350
@Omikoshi78
@Omikoshi78 Рік тому
Ability to break down complex topic is such an underrated super power. Amazing job.
@rohanchess8332
@rohanchess8332 10 місяців тому
How did you condense so many pieces of information in such a short time? This video is on a next level, I loved it!
@tongluo9860
@tongluo9860 Рік тому
Great explanation of the key concept of position encoding and self attention. Amazing you get the gist covered in less than 10 minutes.
@patpearce8221
@patpearce8221 Рік тому
@Dino Sauro tell me more...
@patpearce8221
@patpearce8221 Рік тому
@Dino Sauro thanks for the heads up
@an-dr6eu
@an-dr6eu Рік тому
She has one of the wealthiest company on earth providing her resources. First hand access to engineers, researchers, top notch communicators and marketing employees.
@michaellavelle7354
@michaellavelle7354 11 місяців тому
@@an-dr6eu True, but this young lady talks a mile-a-minute from memory. She's knows it cold regardless of the resources at Google.
@robchr
@robchr 2 роки тому
Transformers! More than meets the eye.
@suomynona7261
@suomynona7261 Рік тому
😂
@Marcoose81
@Marcoose81 Рік тому
Transformers! Robots in disguise!
@DomIstKrieg
@DomIstKrieg Рік тому
Autobots wage their battle to fight the evil forces of the Decepticons!!!!!
@mieguishen
@mieguishen Рік тому
Transformers! No money to buy…
@05012215
@05012215 Рік тому
Oczywiście
@dj67084
@dj67084 Рік тому
This is awesome. This has been one of the best overall breakdowns I've found. Thank you!!
@rajqsl5525
@rajqsl5525 5 місяців тому
You have the gift of making things simple to understand. Keep up the good work 🙏
@maayansharon280
@maayansharon280 Рік тому
This is a GREAT explanation! please lower the background music next time it could really help. thanks again! awesome video
@erikengheim1106
@erikengheim1106 2 місяці тому
Thanks you did a great job. I spent some time already looking at different videos to capture the high level idea of what transformers are about and yours is the clearest explanation. I actually do have an educational background in neutral networks but don't go around remembering every details or the state of the art today so somebody removing all the unessesary technical details like you did here is very useful.
@dylan_curious
@dylan_curious Рік тому
This is such an informative video about transformers in machine learning! It's amazing how a type of neural network architecture can do so much, from translating text to generating computer code. I appreciate the clear explanations of the challenges with using recurrent neural networks for language analysis, and how transformers have overcome these limitations through innovations like positional encodings and self-attention. It's also fascinating to hear about BERT, a popular transformer-based model that has become a versatile tool for natural language processing in many different applications. The tips on where to find pertrained transformer models and the popular transformers Python library are super helpful for anyone looking to start using transformers in their own app. Thanks for sharing this video!
@trushatalati5596
@trushatalati5596 Рік тому
This is a really awesome video! Thank you so much for simplyifying the concepts.
@noureldinosamas2978
@noureldinosamas2978 Рік тому
Amazing video! 🎉 You explained that difficult concepts of Transformers so clearly and made it easy to understand. Thanks for all your hard work!🙌👍
@pumbo_nv
@pumbo_nv 9 місяців тому
Are you serious? The concepts were not really explained. Just a summary of what they do but not how they work behind the scenes.
@axscs1178
@axscs1178 4 місяці тому
No.
@mfatal
@mfatal Рік тому
Love the content and thanks for the great video! (one thing that might help is lower the background music a bit, I found myself stopping the video because I thought another app was playing music)
@PaperTools
@PaperTools Рік тому
Dale you are so good at explaining this tech, thank you!
@luis96xd
@luis96xd Рік тому
Amazing video! Nice explanation and examples 😄👍 I would like to see more videos like this and practices ones
@Jewish5783
@Jewish5783 Рік тому
i really enjoyed the concepts you explained. simple to understand
@harshadfx
@harshadfx 9 місяців тому
I have more respect for Google after watching this Video. Not only did they provided their engineers with the funding to research, but they also let other companies like OpenAI to use said research. And they are opening up the knowledge for the general public with these video series.
@reddyvarinaresh7924
@reddyvarinaresh7924 2 роки тому
I loved it and very simple ,clear explanation.
@user-wr4yl7tx3w
@user-wr4yl7tx3w Рік тому
Wow, this is so well explained.
@TallesAiran
@TallesAiran Рік тому
I love how to simplify something so complex, thank you so much Dale, the explanation was perfect
@decepticon-barricade934
@decepticon-barricade934 Рік тому
how did you do that
@nahiyanalamgir7056
@nahiyanalamgir7056 Рік тому
@@decepticon-barricade934 This one? Just type ":" (colon) followed by "thanksdoc" and end it with another colon. I can add other emojis like 🤟too!
@decepticon-barricade934
@decepticon-barricade934 Рік тому
@@nahiyanalamgir7056 it needs desktop UKposts i think
@nahiyanalamgir7056
@nahiyanalamgir7056 Рік тому
@@decepticon-barricade934 Apparently, it does. When will these apps be consistent across devices and platforms?
@decepticon-barricade934
@decepticon-barricade934 Рік тому
@@nahiyanalamgir7056 thanks though
@shravanacharya4376
@shravanacharya4376 2 роки тому
So easy and clear to understand. Thanks
@MaxKar97
@MaxKar97 Місяць тому
Nice amount of info parted in this video. Very clear info on what Transformers are and what made them so great.
@rembautimes8808
@rembautimes8808 2 місяці тому
This is a very well produced video. Credits to the presenter and those involved in production with the graphics
@bondsmagi
@bondsmagi 2 роки тому
Love how you simplified it. Thank you
@luxraider5384
@luxraider5384 Рік тому
It s so simplified that you can t understand anything
@junepark1003
@junepark1003 5 місяців тому
This is one of the best vids I've watched on this topic!
@todayu
@todayu Рік тому
This was a really, really awesome breakdown 👏🏾
@JohnCorrUK
@JohnCorrUK Рік тому
Excellent presentation and explanation of concepts
@Daniel-iy1ed
@Daniel-iy1ed Рік тому
Thank you so much. I really needed this video, other videos were just confusing
@SeanTechStories
@SeanTechStories Рік тому
That's a really good high-level explanation!
@CarlosRodriguez-mv8qi
@CarlosRodriguez-mv8qi Рік тому
Charm, intelligence and clarity! Thanks!
@touchwithbabu
@touchwithbabu Рік тому
Fantastic!. Thanks for simplifying the concept
@hallucinogen22
@hallucinogen22 3 місяці тому
thank you! I'm just starting to learn about gpt and this was quite helpful, though I will have to watch it again :)
@DeanRGAnderson
@DeanRGAnderson Рік тому
This is an excellent video introduction for transformers.
@RobShuttleworth
@RobShuttleworth 2 роки тому
The visuals are very helpful. Thanks.
@googlecloudtech
@googlecloudtech 2 роки тому
You're very welcome!
@sorbethyena3828
@sorbethyena3828 2 роки тому
Informative! Thank you
@ganbade200
@ganbade200 2 роки тому
You have no idea how much time I potentially have saved just by reading your blog and watching this video to get me up to speed quickly on this. "Liked" this video. Thanks
@ansumansamal3767
@ansumansamal3767 2 роки тому
Where is optimus prime?
@alwaysabiggafish3305
@alwaysabiggafish3305 11 місяців тому
He's on the thumbnail...
@ankitnmnaik229
@ankitnmnaik229 11 місяців тому
He will be in theaters in June 9... Transformers : Rise of breasts..
@captainbob6680
@captainbob6680 11 місяців тому
😂😂😂😂
@yomajo
@yomajo 11 місяців тому
Where are robotaxis?
@yeoj_maximo1122
@yeoj_maximo1122 10 місяців тому
We got lied to
@walterppk1989
@walterppk1989 2 роки тому
Hi Google! First of all, thank you for this wonderful video. I'm working on a multiclass (single label) supervised learning that uses Bert for transfer learning. I've got about 10 classes and a couple hundred thousand examples. Any tips on best practices (which Bert variants to use, what order of magnitude of dropout to use if any)? I know I could do hyperparameter search but that'd probably cost more time and money than I'm comfortable with (for a prototype), so I'm looking to make the most out of my local Nvidia 3080.
@bobdillan5761
@bobdillan5761 Рік тому
super well done. Thanks for this!
@NicolasHart
@NicolasHart 4 місяці тому
so super helpful for my thesis, thank u
@danielchen2616
@danielchen2616 Рік тому
Thanks for your hard work.This video is very helpful!!!
@shailendraburman
@shailendraburman 2 роки тому
Simply loved it!
@josedamiansanchez9874
@josedamiansanchez9874 Рік тому
Amazing explanation!
@jsu12326
@jsu12326 2 місяці тому
wow, what a great summary! thanks!!!
@bingochipspass08
@bingochipspass08 2 роки тому
Very well explained.. This really is a high level view of what Transformers are, but it's probably enough to just get your toes wet in the field!
@barbara1943
@barbara1943 4 місяці тому
Very interesting, informative, this added perspective to a hyped-up landscape. I'll admit, I'm new to this, but when I hear "pretrained transformer" I didn't even think about BERT. I appreciate getting the view from 10,000 feet.
@JayantKochhar
@JayantKochhar Рік тому
Positional Encoding, Attention and Self Attention. That's it! Really well summarized.
@zacharythomas5046
@zacharythomas5046 Рік тому
Thanks! This is a great intro video!
@akashrawat217
@akashrawat217 Рік тому
Such a simple yet revolutionary 💡idea
@mohankiranp
@mohankiranp 7 місяців тому
Very well explained. This video is must watch for anyone who wants to demystify the latest LLM technology. Wondering if this could be made into a more generic video with a quick high-level intro on neural networks for those who aren't in the field. I bet there are millions out there who want to get a basic understanding of how ChatGPT/Bard/Claude work without an in-depth technical deep dive.
@arpitrawat1203
@arpitrawat1203 2 роки тому
Very well explained. Thank you.
@janeerin6918
@janeerin6918 7 місяців тому
OMG the BEST transformers video EVER!
@VaibhavPatil-rx7pc
@VaibhavPatil-rx7pc Рік тому
Excellent explanation i ever seen, recommending everyone's this link
@ayo4757
@ayo4757 Рік тому
Soo cool! Great work
@ZeeshanAli-ck3ue
@ZeeshanAli-ck3ue Рік тому
very well explained.👍
@sun-ship
@sun-ship 2 місяці тому
Easiest to understand explaination ive heard so far
@shivangsharma599
@shivangsharma599 Рік тому
Super Explanation!!
@EranM
@EranM Рік тому
I knew little on transformers before this video. I know little on transformers after this video. But I guess in order to know some, we'll need a 2-3 hours video.
@JG27Korny
@JG27Korny 5 місяців тому
Very informative video. Thank you!
@xiongjiedai8405
@xiongjiedai8405 Рік тому
Very good lecture, thanks!
@WalterReade
@WalterReade 2 роки тому
Nicely done. Very helpful. Thanks!
@myt97
@myt97 Рік тому
Great video. Thank you!
@amimegh
@amimegh Рік тому
NICE SUPERB PRESENTATION
@probablygrady
@probablygrady 11 місяців тому
phenomenal video
@gammacubed
@gammacubed 4 місяці тому
Amazing video, thank you so much!
@takeizy
@takeizy Рік тому
Very impressive video. Thanks for the way you shared information via this video. Reference your video timeline 05:05, how you created such a video, please.
@maxkhan4485
@maxkhan4485 Рік тому
Thanks! Great video.
@anshulchaurasia8762
@anshulchaurasia8762 Рік тому
Simplest Explanation ever
@theguythatcoment
@theguythatcoment Рік тому
do transformers learn the internal representation one language at a time or all of them at the same time? I remember that Chomsky said that there's no underlying structure to language and that for every rule you try to make you'll always find an edge case that contradicts the rule.
@badrinair
@badrinair Рік тому
Thank you for sharing
@KulbirAhluwalia
@KulbirAhluwalia Рік тому
From 5:28, shouldn't it be the following: "when the model outputs the word “économique,” it’s attending heavily to both the input words “European” and “Economic.” "? For européenne, I see that it is attending only to European. Please let me know if I am missing something here. Thanks for the great video.
@Christakxst
@Christakxst Рік тому
Thanks, that was very interesting
@hom01
@hom01 Рік тому
this is brilliant
@robertabitbol6454
@robertabitbol6454 Рік тому
You have actually given the BEST explanation on Neural Machine Translation that I read so far but you are missing a few elements
@robertabitbol6454
@robertabitbol6454 Рік тому
But your explanations, your analyses and your delivery are excellent. You're definitely a great communicator and teacher.
@robertabitbol6454
@robertabitbol6454 Рік тому
Actually Google and others have an algo they're not interested in sharing and I pretty much know what it is. I am working with my programmer on the coding of my new app, the revolutionary Universal Sentence builder and the Universal Dictionary and I keep adding and changing stuff to simplify the concept and I push at a later date the programming of my Sentence Analyser app. It is like most of my apps a simple (and brilliant concept) coded with very few lines of code.
@robertabitbol6454
@robertabitbol6454 Рік тому
You know Alfred Hitchcock was always adapting into the screen his scenario never changing anything not even a comma while Francis Ford Copolla (The Godfather) was doing the opposite: They say that his script was like a newspaper that had new contents every day. Well I am more like Copolla with my apps. I change stuff all the time and I usually make my programmers go crazy. It's a good sign. :-) Mind you I don't know if one can do like Hitchcock with an app. Come up with a definite version once and for all. This would be quite an achievement!
@robertabitbol6454
@robertabitbol6454 Рік тому
In the case of my Universal Sentence builder, the main task was to process the data entered by the user and we've been at it since July 2022. :-) It's either I am dumb or it is a complex task. Actually it is the latter for I have started with French, this langage being the most complex in the world. The good news is I am sure I will be imitated but you can rest assured that my imitators will also have a jolly hard time with French :-)
@rodeoswing
@rodeoswing 6 місяців тому
Great video for people who are curious but don’t really want to (or can’t) understand how transformers actually work.
@massimobuonaiuto8753
@massimobuonaiuto8753 Рік тому
great video, thanks!
@user-or7ji5hv8y
@user-or7ji5hv8y 2 роки тому
Great video.
@GubeTube19
@GubeTube19 Рік тому
10/10. Very helpful
@wiclcoocoo
@wiclcoocoo Місяць тому
a very nice video. thanks
@Mariouigi
@Mariouigi Рік тому
crazy how things have changed so much
@EduardoOviedoBlanco
@EduardoOviedoBlanco Рік тому
Great content 👍
@intekhabsayed4316
@intekhabsayed4316 2 місяці тому
Good(Pro) Explanation.
@RonaldMorrissetteJr
@RonaldMorrissetteJr Рік тому
When I saw this title, I was hoping to better understand the mathematical workings of transformers such as matrices and the like. Maybe you could do a follow-up video explaining mathematically how transformers work. thank you for your time
@gerardovalencia805
@gerardovalencia805 2 роки тому
Thank you
@cassianocominetti7784
@cassianocominetti7784 4 місяці тому
Amazing!
@aGj2fiebP3ekso7wQpnd1Lhd
@aGj2fiebP3ekso7wQpnd1Lhd Рік тому
Fantastic video
@MichaelToop
@MichaelToop Рік тому
Great video. Thx.
@PrakashNarayananP
@PrakashNarayananP 2 роки тому
Well done explanation.
@softcoda
@softcoda Рік тому
Wowww….thanks for clarifying my confusion.
@TechNewsReviews
@TechNewsReviews 7 місяців тому
woww, she's good at explaining things
@directorblue
@directorblue Рік тому
Well done
@hughesadam87
@hughesadam87 Рік тому
Thank you!
@samsont81
@samsont81 Рік тому
You are amazing!
@younessnaim1849
@younessnaim1849 Рік тому
Beyond the great content and delivery, I loved your French accent ... ;)
@JosephHenzi
@JosephHenzi Рік тому
I'll jump on where others are doing the same - would love advice for someone who understands half the concepts that are alluded to as complex naturally and the innovation feels obvious I'm unsure how to break into the space without some guidance or connection between having exactly that great natural grasp but wildly anxious that language and logic are strengths and math is a mental turn off. For someone needing that type of translation/guide where my approach is language usage & finer cues what is the key terms to get to that understanding? Hate being fascinated and all the tools to play in this space and being unable to start because how I approach topics so welcome any advice.
@meepk633
@meepk633 Рік тому
Just go to school.
@Maisonier
@Maisonier Рік тому
Amazing video, thank you ... can you use transformers to detect patterns in random data that which is supposedly unpredictable, like weather or stocks?
@Happypast
@Happypast Рік тому
the unpredictability of stuff like weather and stocks has to do with the fundamental underlying nature of those phenomena so I would bet no.
@johnbarbuto5387
@johnbarbuto5387 Рік тому
An excellent video. I wonder if you can comment on "living the life" of a transformers user. For example, in another video by another UKpostsr I heard the sentiment that being an AI person in this era means constant - really constant - study. That may not be the lifestyle that everybody wants to adopt. I'm a retired neurologist and vice president of the faculty club at my state university. What interests me these days is how students "should" be educated in this era. And, at the end of the day, one of the critical aspects of that is matching individual human brains - with their individual proclivities - with the endless career opportunities of this era. So, I'm trying to gather perspectives (aka "data") on that topic. Maybe you could make some kind of video about it. Please do!
@LimabeanStudios
@LimabeanStudios Рік тому
I think the most important thing is that students are simply encouraged to use these tools. It's pretty hard to get a realistic grasp of the capabilities without really pushing the systems. The idea about needing to do constant research is interesting, and I think it's something that a person CAN do (the rest of my life probably lmao) but I think simply adopting the tools is all that will effectively matter. It's too early to be much more specific sadly. When it comes to younger education then we definitely need to be putting more focus on skills and behaviors instead of knowledge.
@bobsalita3417
@bobsalita3417 2 роки тому
Well written script. Appreciated.
@amortalbeing
@amortalbeing 6 місяців тому
Thanks a lot.
@tusharjamwal
@tusharjamwal 10 місяців тому
How did you sync your talking cadence to the background music?
@horus4862
@horus4862 Рік тому
That was really good. Thank you
Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!!
36:15
StatQuest with Josh Starmer
Переглядів 548 тис.
НЕОБЫЧНЫЙ ЛЕДЕНЕЦ
00:49
Sveta Sollar
Переглядів 6 млн
GPT-4o - Full Breakdown + Bonus Details
18:43
AI Explained
Переглядів 29 тис.
Transfer learning and Transformer models (ML Tech Talks)
44:59
TensorFlow
Переглядів 113 тис.
The most important AI trends in 2024
9:35
IBM Technology
Переглядів 183 тис.
AI Pioneer Shows The Power of AI AGENTS - "The Future Is Agentic"
23:47
Matthew Berman
Переглядів 368 тис.
Transformers for beginners | What are they and how do they work
22:48
Code With Aarohi
Переглядів 26 тис.
The math behind Attention: Keys, Queries, and Values matrices
36:16
Serrano.Academy
Переглядів 193 тис.
Let's build GPT: from scratch, in code, spelled out.
1:56:20
Andrej Karpathy
Переглядів 4,2 млн
Transformers for beginners | What are they and how do they work
19:59
AssemblyAI
Переглядів 141 тис.
Fine-tuning Large Language Models (LLMs) | w/ Example Code
28:18
Shaw Talebi
Переглядів 220 тис.
Эволюция телефонов!
0:30
ТРЕНДИ ШОРТС
Переглядів 5 млн
Xiaomi Note 13 Pro по безумной цене в России
0:43
Простые Технологии
Переглядів 301 тис.
❌УШЛА ЭПОХА!🍏
0:37
Demin's Lounge
Переглядів 306 тис.
Самый маленький Iphone в мире!📱(@ghoul.arena)
0:22
Взрывная История
Переглядів 353 тис.
#Shorts Good idea for testing to show.
0:17
RAIN Gadgets
Переглядів 3,2 млн