Tesla Dojo AI Supercomputer Deep Dive and Analysis
Переглядів 41,578
День томуmore-moore.com : Sign up to the More Than Moore Newsletter
A deeper look into how Tesla's Dojo microarchitecture and system level design is constructed.
[00:00] Intro
[00:56] More Than Moore Newsletter
[01:21] Part 1: Training vs Inference
[03:48] Part 2: Dojo D1 Chip and Core
[11:21] Part 3: Packaging and Tiles
[15:50] Part 4: Custom Network Silicon
[19:13] Part 5: The Tesla Dojo ExaPod
[23:40] Part 6: Comparison to Cerebras
[25:32] Part 7: Tesla AI Day 2022
[26:59] Thoughts on Tesla FSD
[29:26] Cat Tax
-----------------------
more-moore.com : Sign up to the More Than Moore Newsletter
/ techtechpotato : Patreon gets you access to the TTP Discord server!
Follow Ian on Twitter at / iancutress
Follow TechTechPotato on Twitter at / techtechpotato
If you're in the market for something from Amazon, please use the following links. TTP may receive a commission if you purchase anything through these links.
Amazon USA : geni.us/AmazonUS-TTP
Amazon UK : geni.us/AmazonUK-TTP
Amazon CAN : geni.us/AmazonCAN-TTP
Amazon GER : geni.us/AmazonDE-TTP
Amazon Other : geni.us/TTPAmazonOther
-----------------------
Welcome to the TechTechPotato (c) Dr. Ian Cutress
Ramblings about things related to Technology from an analyst for More Than Moore
#techtechpotato #Tesla #dojo
------------
More Than Moore, as with other research and analyst firms, provides or has provided paid research, analysis, advising, or consulting to many high-tech companies in the industry, which may include advertising on TTP. The companies that fall under this banner include AMD, Armari, Facebook, IBM, Intel, Linode, MediaTek, NordPass, Qualcomm.
@TechTechPotato Рік тому
more-moore.com : Sign up to the More Than Moore Newsletter [00:00] Intro [00:56] More Than Moore Newsletter [01:21] Part 1: Training vs Inference [03:48] Part 2: Dojo D1 Chip and Core [11:21] Part 3: Packaging and Tiles [15:50] Part 4: Custom Network Silicon [19:13] Part 5: The Tesla Dojo ExaPod [23:40] Part 6: Comparison to Cerebras [25:32] Part 7: Tesla AI Day 2022 [26:59] Thoughts on Tesla FSD [29:26] Cat Tax
@Zadagu Рік тому
Thanks, I absolutly like your deep dives into architectures, like IBM z16 and Cerebras. Would love to see more content like this.
@EdPin_ Рік тому
Dr Ian Cutress. Here your title punches pretty hard, i had to pause from time to time to take a breath and digest. Excellent material 🙂
@DarkArtGuitars Рік тому
This is the type of content I'm subscribed for. Just started on the VLSI track at uni so it is super interesting to see what is out there.
@steveseeger Рік тому
Great review and perfect timing for a refresher! Hope you do a follow-on in October if Tesla releases enough new details.
@shanecook9653 Рік тому
Really nice overview. Hopefully someone from Tesla send you an invite!!!
@techno_ismail1458 5 місяців тому
Great insight into this architecture. I’ve got an exam tomorrow focusing on the DOJO Tesla, and this video gave a great summary of the main points.
@jurepecar9092 Рік тому
One of the subtopics you could explore a bit more in depth are number formats. Take a look at Gustafson's work on unums and posits, they should be the "one format to rule them all" going forward.
@TechTechPotato Рік тому
Vividsparks has reached out. But yeah, a look into number formats is on my to-do list.
@ophello Рік тому
Should Tesla have used posits for dojo instead?
@Sadiinso Рік тому
I remember reading a paper which described the fact that posit arithmetic was less efficient to implement in hardware compared to IEEE754 fp formats (on FPGAs at least). Although the article did not discuss smaller more efficient formats such as BF16 or FP16, its conclusion could explain why this format was not used by tesla.
@leeterthanyou Рік тому
29:28 the eagle-eyed among you will notice the cat tax has been doubly paid
@mibo747 Рік тому
Thanks for SUPER DETAILED presentation
@gj1234567899999 Рік тому
Thank you for this thorough and non biased analysis!!!
@basembarakat Рік тому
Thank you so much for this deep dive and especially for the putting the effort into comparing Dojo with Cerebras
@chrisfisher6700 Рік тому
Brilliant walkthrough. Thank you for the detailed presentation.
@PlanetFrosty Рік тому
Excellent review of the the entire process Tesla has gone through in creation of Dojo. We’ve been working on optical processing for 20+ years in a hybrid format and it interesting to see Tesla’s focused approach. Our work is hybrid technology, but the optical cores reduce power and provide us with different approach to similar AI problems.
@AllYouWantAndMore Рік тому
Excited to see your breakdown after Ai Day 2
@doobiedoo5450 9 місяців тому
Excellent vid! Thank you!
@wraith6766 Рік тому
Great video. Very insightful 👏
@JamyRyals Рік тому
Really enjoyed the video!
@m_sedziwoj Рік тому
It is great that you make this video, and I think you at Tesla AI Day 2 would be align with they goals, because not all people would know about this event. And it would be great, if they gives more numbers, to make deeper comparison between Tesla solution, and not only Cerebras but other systems (as Tesla is using Nvidia GPU cluster today) too.
@pandiyanp5816 8 місяців тому
impressive sir!
@acasccseea4434 Рік тому
thankyou alexia, for reading the excellent writeup
@edahmed7 Рік тому
Awesome video thanks doc😊
@RoundSparrow Рік тому
Great video, thank you for making this.
@kalebbruwer Рік тому
No way you watched it yet
@RoundSparrow Рік тому
@@kalebbruwer I did, i commented about 2/3 into it.
@kalebbruwer Рік тому
@@RoundSparrow My bad
@Karthig1987 Рік тому
Awesome Video doctor.
@simolatham03 Рік тому
Good work DIC!
@mrjean9376 Рік тому
DEEP DIVE contents Always be in my favorites in every youtube science channels, please do more of this awesome topics this is very interesting to watch and learn
@mrjean9376 Рік тому
I'm subbing ♥️
@latemhh5577 Рік тому
Thank you
@nathanprice4568 6 місяців тому
Good job
@DihelsonMendonca Рік тому
⚠️ Hey, you found an excellent way to hide your microphone wearing a black shirt. Simple and functional. I already tried many solutions to hide my mic, but the best one was to use a mic like yours, pointed to my mouth, and eliminate all reverb from the room. Great 👍 🙏❤️
@opius1199 Рік тому
Impressive!
@EyesOfByes Рік тому
One word: insane
@mansemi5480 Рік тому
Interesting designs. Tesla uses NPUs(kinda stripped version of GPU in some manner) for FSD chips but uses CPU cores for training. Thank you for a nice analysis. I'm really curious how their training codes are implemented.
@Veptis Рік тому
Will there be an update with what you were able to learn at the event?
@woolfel Рік тому
thanks for the detailed breakdown. It seems similar to Cerebras where cores have small SRAM and designed for stream processing. Makes sense given Tesla is moving to training on video streams, instead of individual frames from separate cameras. It would be interesting to see a comparison of how TPU v3 handles streaming video training to DOJO. Google has been training on video for several years now. autonomous taxi has already gone live in ShenZhen china and their FSD seems more advanced that the current Tesla FSD beta.
@linmal2242 Рік тому
Yes but is it better to use Brainchip's ultra power efficient Akida chips?
@stephenpalmoore147 10 місяців тому
Do you think Dojo is ready for production as they recently stated?
@J3llyBean423 11 місяців тому
AVP yeah / now I'm trying t pressure freeze diamonds / so the chips don't get hot making them //
@thomasruwart1722 Рік тому
It would be interesting to see how the Dojo chip can do computational fluid dynamics. That would help with weather forecasts, for example. Then again, an AI-based approach to weather forecasting might be more appropriate for the Dojo chip.
@RoundSparrow Рік тому
@27:10 - FSD in passive mode. Pothole mapping would be interesting, especially of shock movement data is available.
@m_sedziwoj Рік тому
I think they focusing not to not fail at weird intersections ;) As they did talk in past, they pulling data they want to optimize in given moment.
@marcusk7855 Рік тому
I wonder if Tesla is open sourcing Dojo?
@gunnargu Рік тому
They should consider making idle time on theirs public, or make a small one for public use. Mostly for getting devs experience and or excited with the architecture and then maybe hire them :D
@tdwebste Рік тому
Love to drive a Tesla through white outs. That should give them some interesting events. Using hydro poles as markers and feeling road tracks, because it is impossible to see the road.
@charlesrosenbauer3135 Рік тому
The idea of each chip being a grid of cores, and the whole supercomputer being a grid of those chips that link their individual grids together is very similar to what Adapteva originally was planning for their chips.
@Steamrick Рік тому
What's the cat tax song? (Sorry if you've mentioned the name in a past video - I don't see any mention in this video or the description)
@TechTechPotato Рік тому
ukposts.info/have/v-deo/an5hqKOemKtmpmg.html
@miles2378 Рік тому
Cat pictures infest the internet and actual cats plage computer users.
@Quarky_ Рік тому
Is this the first major RISC-V chip? I know there are many micro-controllers, this would probably fall under accelerators.
@TechTechPotato Рік тому
It's not RISC-V. Don't get RISC confused with RISC-V.
@Quarky_ Рік тому
@@TechTechPotato Ah, thanks. It's some kind of TESLA version of RISC
@Kimf87 Рік тому
First time listener here. You have a plesant voice.
@WeirdSeagul Рік тому
you are the type of journalist that Elon would want at the event that can read the data with understanding compared to places like CNN Fox etc
@franciscolodix1214 Рік тому
Hi Ian, long follower here !!! Just as feedback not criticism, but the audio quality is not great haha
@JinKee Рік тому
The architecture looks a lot like the CPU in a Cyberdyne Systems T-800 model 101 Terminator
@KieferNguyen Рік тому
wait did you say telsa leading the way in automous self driving??? Hello?? What has Google been doing with with Waymo???
@Redstoner2b2t Рік тому
google and what mass-deployable neural net?
@bryce.ferenczi Рік тому
I'm certainly interested to see some kind of real comparison between this and the Cerebras. If they were purely running convnets, punishing FMA ops, branch prediction and scalar units are wasted silicon. Even nvidia's "sparse" tensor engine is wasted space for most applications, a mux choosing the non-zero element of a pair of contiguous elements is just a neat trick to increase theoretical numbers that doesn't really help the majority of workflows. You have two non zero elements next to each other? Sorry, you can't use sparse op/its output is malformed.
@adamrak7560 Рік тому
The sparse tensor stuff is good for slightly sparse inferencing (if you have trained for sparsity), but useless for training. You can even train the network to make sparsity in the pattern that the sparse tensor engine likes.
@Cinnabuns2009 Рік тому
I would think you'd put out a video like this on the "day of" or "week of" Tesla's full self driving release which was promised 5ish years ago? Is it out yet?
@grambo1980 Рік тому
This musta been one Hellva script to write. AI day 2 next week!!
@stevesloan6775 Рік тому
I feel the missing keywords are "Organic electron flow"
@Arowx Рік тому
Great info. Q: Who would you want to build a next gen Personal Computer, Cerbras or Tesla?
@kevin-jm3qb Рік тому
kinda looks like tenstorrents architecture :P
@kevinkrewell5849 Рік тому
The Dojo system performance and efficiency is heavily dependent on the software Tesla can develop. Chip specs are really meaningless unless they build the right software.
@DihelsonMendonca Рік тому
I have no doubt they can do it, do you ? 🙏👍😄
@cbuchner1 Рік тому
But isn‘t it so that DOJO might come online with so much delay that off the shelf alternatives like nVidia Hopper would already have caught up in performance?
@TechTechPotato Рік тому
Dojo is meant to be intimitely optimized for Tesla workloads. One thing you can't easily get over is chip to chip connectivity. Nvidia hasn't scaled that with Hopper.
@cbuchner1 Рік тому
@@TechTechPotato NVLink for Chip to Chip interconnects and Mellanox Infiniband to connect the nodes isn‘t too bad. Depends on the exact workloads whether this is a limiting factor or not.
@ChrispyNut Рік тому
Tesla going this route, how long until we have "Captcha" from Google showing us driving incident footage and we have to input what we'd do, in order to "prove we're human" when actually we're just teaching Google how to drive.
@m_sedziwoj Рік тому
Problem with google solution is "good data", even if you pay and train people they make mistakes. This is why one of big moment of Tesla training was auto label, which cut time and errors.
@NarekAvetisyan Рік тому
My question is: Wouldn't they also benefit from analogue chips for machine learning that Veritasium has talked about? With the recent rise of analogue computers and what not.
@TechTechPotato Рік тому
That Veritasium video was kinda laughed at in the industry. He failed to make the distinction between chips used for training and chips used for inference. Mythic are building chips that do inference, whereas Dojo is training.
@_DarkEmperor Рік тому
1.25MB of SRAM is not a cashe, it is more like a scratchpad.
@vikaspoddar9456 Рік тому
Sir, will you teach us about microprocessor, microcontroller, gpu and other things such as logic unit, other specialized hardware ASIC and FPGAs
@Adam-zd2bk Рік тому
Simple
@visvamba Рік тому
Big fan of the channel, but to be honest though this was really in-depth, it felt kinda rushed and hard to follow. Had to pause and repeat quite a few times.
@TechTechPotato Рік тому
Check out the written version!
@andytroo Рік тому
love the overview. would Tesla's approach of silicon on wafer, as compared to cerebus direct wafer scale chip, limit power, as there is an extra layer to transfer power/heat through?
@UpLateGeek Рік тому
You gotta wonder how cost effective it is to roll your own vs just buying something from a company that specialises in those things.
@kazedcat Рік тому
It depends on how large you are scaling up. If you are at a warehouse level meaning your computing needs can fill up an entire warehouse then it starts to make sense to roll out your own custom microchips. Tesla is already at that level. Currently they are running three warehouse size clusters of Nvidia GPU and they recently added a new one. Their projection is that they need to 10X their current computing power so they really need custom supercomputer to reduce their computing cost.
@Alex.The.Lionnnnn Рік тому
Would it be possible for Tesla cars to all do a small amount of training as they go and transmit the data back to the data centre to make the whole fleet of Teslas worldwide help continuously help refine the algorithm?
@TechTechPotato Рік тому
That's what they do - transmit the interesting clips back and use it for training in the datacenter. Training doesn't happen on the car. Same reason why you don't render graphics on a CPU : 100000x difference in perf and perf/watt
@Alex.The.Lionnnnn Рік тому
@@TechTechPotato Ok I figured they'd be doing that, but wondered if perhaps there would be a certain critical mass of Teslas all processing that it would become feasible. Thanks for the response!
@cornbreadcuban5456 Рік тому
This is like the third video this week that the names Tesla as “ leading the way in autonomous driving”. What do they have that no one else has again other than broken promises?
@gptinsights Рік тому
Making videos about tesla guarantees views for UKpostsrs. The people love tesla content. That the company is full of shit doesn't matter for the ad revenue
@jasonosmond6896 Рік тому
@@gptinsights Dr. Cutress is also angling for that A.I. Day invite, so the logic is pretty simple: Make a Tesla Dojo video, include a note at the end that he's trying to get a A.I. Day invite, get 2 million views from extremely online people who do nothing but watch and talk about Tesla all day, and hope enough people spam Elon's Twitter feed with links to the video and he extends the invite. I'm not being critical of Dr. Cutress here, it's a valuable video in its own right, but Elon only seems to interact with the world through his Twitter feed, so this video is what the meta requires.
@gptinsights Рік тому
@@jasonosmond6896 yeah I don't blame him It'll make his channel more popular. But he might lose some credibility for it
@hypersonicmonkeybrains3418 Рік тому
Dumb question can they use all the waste heat to drive a steam turbine and generate electricity?
@daemonbyte Рік тому
Why is it everyone in the media keeps saying tesla is leading the way when they only have a fancy cruise control and Mercedes has level 4 self driving and Weemo has taxis literally already driving themselves?
@vincentwhite7693 Рік тому
Lol
@LoveBbyJay Рік тому
Not to be an a$$ but Tesla has a horrible record with their self driving cars, and other car manufacturers already have more certifications for FSD tech in their vehicles across the world whereas Tesla is loosing some of their FSD certifications... Tesla was the first longish range mass market EV that doesn't make them the best and that's becoming more and more apparent as the years/months go by.....
@SalvatorePellitteri Рік тому
I want more silicon details!!!!! Muuhhahahah
@No_Free_Lunch_Today Рік тому
Seems like you didn't want to do it. it was interesting nonetheless! See you on the next one.
@TechTechPotato Рік тому
Seems like I didn't want to do it? What?
@No_Free_Lunch_Today Рік тому
Dojo deep dive. Maybe it just me, just seem to have little less enthusiasm. As before still enjoyed watching.
@unvergebeneid Рік тому
Everyone and their mom is designing their own processors these days...
@donaldendsley6199 Рік тому
While there is a lot I like about Tesla's approach, I can't help but feel like they (and other self driving tech companies) are still trying to brute force the problem. I think the decision matrix is too large to really be able to use this approach only. I feel like they are going to need a way to vastly speed up "training." Conceptually I'm thinking they are in the stage of where path tracing was a decade or so ago, and they need tech akin to DLSS and other tech that is making "real time" path tracing possible. Yes, I know those use "AI" and I'm not saying to use AI to solve an AI problem, I'm trying to say there is a related tech that actually needs to be developed before we can solve the FSD problem. Call it Artificial Imagination, basically a way to pare down the problem in a way that makes some training possible on the in vehicle hardware, or at least fast enough that it could happen over a network in close enough to real time.
@ophello Рік тому
“What’s your minimum specification?” Is that some kind of inside joke for chip experts?
@m_sedziwoj Рік тому
my minimum specification is potato ;)
@ophello Рік тому
Does no one have an answer?
@popcorny007 Рік тому
@@ophello Yes, it's an inside joke. Basically just "in closing", or "I would like to see"
@ophello Рік тому
@@popcorny007 I still don’t get the joke. That’s how the video starts, so I’m not even sure what you’re taking about.
@mdonatas Рік тому
Could you please not add the progress bar to your videos or make them very subtle somewhere in the corner. The green bar was so distracting that I kept staring at it and mindlessly skipping all the interesting content you were presenting. Thanks! 🙏
@sdmarlow3926 Рік тому
Hundreds of billions spent on "training" across the field, and it's not even the right path forward.
@johnsonjjohnson100 4 місяці тому
You are not keeping up. Dojo is a complete flop. Musk fired the head module designer for non-performance. He designed the Dojo module with little to no memory. The idea was that it would make the system faster Turns out AI is a memory pig to work efficiently!!! So at the very least the entire Dojo project will need to be redesigned or scrapped altogether or Maybe repurpose the chips for a huge data center after adding memory The good news is that FSD AND Optimus are NOT dependent on Dojo Our original supercomputer is still the one we lean on for all FSD and Optimus iterations The 10,000 H100 Invidia chips went to our old supercomputer -- NOT Dojo
@TechTechPotato 4 місяці тому
You're commenting on a 16month old video.
@byteme6346 4 місяці тому
If TechTechPotato knew anything, he would have a JOB, not a YT channel.
@TechTechPotato 4 місяці тому
I have a JOB outside YT. I'm very well paid. Google me sometime. ✌️
@oraz. Рік тому
I'd call it a model but I am an ass
@JohnWilliams-gy5yc Рік тому
Wish Elon creates desktop risc-v out of this someday.
@gptinsights Рік тому
since when are you making science fiction videos ;) tesla dojo is prob a fugazi
@PicaDelphon Рік тому
Chip On Wafer, You have a C.O.W., Well time to Milk this C.O.W. And Pet the Tabby boy...
@hamesparde9888 Рік тому
It really triggers me when people talk about GPU SIMD lanes as if they along constitute cores. Like people that will tell you "this GPU has X thousand cores" and you try to tell them that technically it doesn't, but they keep saying it 🙄
@TechTechPotato Рік тому
Good thing these are actual cores, and not simple SIMD. My whole PhD was programming GPUs.
@adamrak7560 Рік тому
Modern NVIDIA GPUs are very good at pretending that those SIMD lanes are separate cores, so good that it has caused us very nasty bugs. (Most of our code was written with awareness of the SIMD lanes, and optimised for them) My PhD was mostly programming GPUs too. (and designing an architecture which looks quite similar to Dojo, to solve the multi GPU scaling issues)
@telesniper2 4 місяці тому
"superscalar multi-threaded organization" for their subprocessor nodes on a transport oriented architecture. That right there tells me they don't know what the f*ck they're doing. I think chip design automation tools have become sophisticated enough we are seeing the rise of the equivalent of "script kiddies" and "code monkies" in chip design. They just click on features they think sound cool based on the buzzword and make the design tools throw them in the final design, LOL
@tommorgan1291 10 місяців тому
No one I know is interested in self drving,nd no one I know plans on using self driving. So quit wasting time and instead build an EV 450 mile range, quick charging, lots of charging stations and priced in the mid twents.
@mhsohel47 Рік тому
Reading slides.
@eclecticmemes Рік тому
Per your last thoughts..... It's been my contention from the beginning that Tesla risks being accused of being "racist", if FSD begins to mimic driving behaviors that are only seen in America in the very risky, high crime neighborhoods. For example, It is normal procedure to never, ever stop at stop signs or traffic lights after 10 pm for example, for fear of having your car hijacked or shot at. If Tesla ever implements algorithms to mimic local driving behavior (which is 100% necessary to Drive in Mumbai, Bangkok, Germany, and Harlem), then that will be the level at which jeers of racism will occur.
@QuikRay Рік тому
AI has no ability to determine context on it's own....a very serious issue and short-coming.
@kazedcat Рік тому
You should learn about AI before you talk about them. Machine learning is all about learning context from a vast amount of data.
@garethevans9789 Рік тому
A car company that is giving the likes of Nvidia and AMD a run for their money... Even Intel with their vast sums of R&D couldn't make Itanium work (but they didn't have TSMC). The power desity is immence (more than most racks in just 2U?).
@ShainAndrews Рік тому
LOL... Oh the ignorant.
@user-rz1vh7go6x Рік тому
넌 뭐하는 얘니?
@jtmcgee Рік тому
I love that EVs are moving into the mainstream but I hate Musk and his pseudo intellectual, fake liberal, kind. My hate for him and those like him drive me away from EVs, etc. I see his name attached and cant bring myself to be interested. I know this video isnt about Musk BUT this is how much I hate him. I dont hate many things, hate is a strong word.
@kazedcat Рік тому
You should touch grass. I hate Putin but I would not late that hate consume my life.
@Alperic27 Рік тому
Blah blah blah … you are out of your depth when talking about ai… stick to hardware dude…
@TechTechPotato Рік тому
How so?
@RogerBarraud Рік тому
What's My Minimum Irritation? Fake accents... 😕 #Grrrrr
@Davethreshold Рік тому
After just watching the NASA DART hit an asteroid, and then stumbling across this, I think almost ANYTHING is possible with Humankind! Plus I'm getting ready to build my Intel 12th Gen Computer. Thank you for your video! ❤🤍💙
Eurovision Song Contest
Переглядів 15 млн
Эстрадный продюсер №1 Перман. Зеленский, Ермак, Ротару у Кадырова, жадина-говядина Порошенко, Галкин
В гостях у Гордона
Переглядів 935 тис.
Immersed
Переглядів 28 млн
Раскосов - продвижение
Переглядів 1,3 млн