Linus holding a 31.000k SSD? This is going to be a wild ride.

One of my favorite things in LTT videos is Alex being worried about jank things that other people do even though he is the jank master himself. He will be worried about other people doing things and do something 10 times worse a minute later. And I am here for it.

i absolutely love this 'we shouldnt be doing this' dynamic that these two have

What you guys should have done to really push the speed, was have it directly feed from your 100gb server connection and see how quickly it can pull your entire test suite. I would love to see this thing install 6tb of steam in 14 seconds.

"Hot plug" refers only to the switch chip itself, m.2 doesn't allow for it. The mechanical interface still has to be designed to ground the SSD before applying power and limit the inrush current.

I love videos with Alex and Linus. Linus loves to do things the janky way, and Alex has an engineering background, so Linus has a hope that Alec will do things the correct way, but then when Alex does thing the janky way like Linus isn't happy, but then things work out, and he's happy again.

For the VFX Editor (this may be a simplistic view) things like Baselight X where uncompressed video with native 8K raster EXRs are used outside of a Proxy workflow for finishing, the sheer size of uncompressed EXRs combined with high frame rate requests now, means your bottle neck is storage speed. There are bespoke Linux appliances that are used for review and feedback for workflows in VFX where high-speed RAID Cards are used for caching these frames. This would be ideal for this purpose

I could totally see this device being used in massive weather simulations where you need to store the values of the atmospheric conditions within individual blocks of data and need the fastest access possible to that data. Being able to store the entirety of the information contained within a storm on a single device would prove invaluable to meteorologists, especially Dr. Orr in Minnesota who's been simulating tornadoes in his supercomputer.

I remember seeing something similar to this way back in the 80's. They were essentially the first SSDs. It was a board that you'd install a bunch of RAM on. Back then RAM didn't come on DIMMs. It was a bunch of socketed IC chips that looked like EPROM. Then the board would be installed in an ISA slot (it was probably EISA, don't remember). There were also no drivers. The board would present itself to the BIOS as a native storage device, kinda like how ST-506 and ATA did. The BIOS would facilitate communication between the OS and the "drive". It was a lot like MFM and RLL drives that had their own expansion cards. This was all pre-Windows and GUI OSes.

Pros of BIOS RAID: - More "natural" path to installing your OS on a RAID volume. - Presents the volume to the OS as a single drive (mostly only significant because of the previous point). Cons of BIOS RAID: - In almost every case, you need drivers to use the volume in your OS. This is a common pain point during OS install. - For consumer boards, it's still just a software RAID (see impact in the next 2 points). - There are almost no performance benefits in regards to CPU usage (tested this many times). - Data throughput isn't any faster than using an OS software RAID (tested this many times). - If you ever move your drive(s) to a different system (especially a different brand) that doesn't use the same BIOS RAID, you can't access the data. OS level RAIDs will always be recognized by the OS (unless you toasted your drives somehow). Do not confuse this for a list comparing hardware vs software RAID because consumer grade BIOS RAID is not hardware RAID.

I can see the utility of this. When I was still at uni (back when the fastest way to transfer data was TAs with USB keys), I worked with a professor on a project where he was using datasets that were about 32GB each, and he was having to go through about 100 to map them out. The software , which he made, required it to be loaded into memory, chucked out and recalled later. So this was on the order of 1 test per night on a (at the time) top of the range dual Xeon system. And I successfully got it up to 2 tests per night, hold your applause. Something like this would have been a Godsend. I mean, I'd have to do more work writing code, so I am glad it wasn't a thing.

5:10 I don't know why but I felt a chill down my spine of someone out there jaw dropping at this zoomed out moment.

I’m not so sure. A ram’s top speed is 20 mph. How could an SSD possibly outclass an animal of such swiftness?

Would love to see this thing loaded purely with Intel optane drives, i.e. 118gb p1600x ngff 2280 drives. It would only be ~2TB, of data, but imagine the iops.

This would be great for a Virtual Computer center - I’m biased towards education - having Lab VMs available on demand across a LAN would be fantastic. I’m only part way through watching as I post this, but I would be very interested in IOPS on a new heavy access load. Aaaaand… you didn’t let me down! You did the IOPS. Writes in a RAID will always cause a performance hit (especially because you won’t be able to use single parity bit and will incur extra cost on >8TB - which this definitely is!), but if I was using this it would be for VMs and Data with a heavier READ profile and it would be pretty cool.

Your videos are really awesome to learn about hard and software. I found alot of interesting stuff in the internet when you were throwing words in the room from which i often never heard of and then made me google it. Keep up the good work

Sabrent really going at it lately first with one of the best options for steam deck now searching out large silly projects to sponsor just to show how much progress they as a company has made is just insane. I swear haven't even heard of them till last year

I'm happy this has come up as a topic of conversation. I've been wanting someone to test SSD speed to say yesteryear RAM versions like DDR3 DDR2 etc. to see if SSDs using swap files are at parity with RAM

how to build worlds most expencive toaster :D

18:30 The Anthony phone-a-friend lifeline is my favorite thing ever

Linus is doing this for more than a decade, you would think his enthusiasm will be gone by now but no, it increased by a ton. This is why LTT is so cool!

I was actually expecting something jank when Alex showed up, but just 21 SSDs on a PCIe port surprisingly tame. With that said, I wonder how much of the performance would be retained if this SSD was used on a PS5.

Machine learning is definitely going to be a big use case. Even sample datasets we use, given for educational purposes and are not meant to be very challenging , are easily 500GB in size sometimes. And they relatively small datasets since they really do not have that many different variables. Now imagine having to handle literally petabytes of data.

Imagine watching this video 10-20 years from now, and you can fit that amount of data in your pocket. Tech is so wild, I'll never get enough.

2:25 "I would suck on your toes for this many drives" don't drop yourself like that Linus 🤣🤣 PS: these jokes come exclusively from love and admiration -please don't drop me from your channel- .

This is perfect for data centers! I would love something like this for computer vision models! But I am sure we need motherboards with a lot of pcie lanes, like on the thread ripper.

Alex just standing in the background is my favorite part about Linus/Alex videos

My first Computer had a Hard drive controller card of course 8 or 16 bit slot... but it's funny we have come full circle...amazing can't wait to add one of these new cards...

its crazy to think just a few years ago Sabrent was the "try it if you want" brand I bought a 240gb drive probably 5 years ago and I was skeptical but damn they are on my list of good brands for M.2s

The X21 is more than just a one-trick pony. The M.2 slot supports a number of other devices. For instance, you could use wireless cards with M.2 A+E converter boards to set each wireless card to a specific wireless channel for spectrum monitoring. There’s even an AI angle here. A company called Axelera makes an M.2 AI Edge accelerator module that could be used by the Apex card at some point as well.

It is interesting that the use has an expiration date, whoever uses this type of equipment in the proposed scenario of continuous use reaches the TBW of the units in a very short time. I remember setting up SLC caches on dedicated servers more than a decade ago, even though it was SLC the replacement cost was already considered at the beginning of the project, even though it was expensive there was no other way in the same cost range

This was a year ago and Linus was right. These cards did catch on a little, and they work. I put another brands PCIe 4.0 card in an older computer, stuck 2 M.2 drives on it, and it works great. I cannot tell they're on a PCIe board and not on the mobo. Worked great!

the insane part is how much of a "great fit" and "bargain price" this could be considered for high-end enterprise and workstation systems working on big datasets. This density of performance and storage without "custom" drives is unrivaled

I'm in love with this thing because of NAS storage stuff. Sure, this card is really expensive and might be overkill but having 21 SSDs gives you a lot in terms of fault tolerance (RAID5 or RAID6) while also consuming less space. I'm excited to see where this is going!

It sound like it dynamically routes pci to where it is needed, It is super cool that it is possible to run any 4 drives at full pci speed or 8 cheaper slower drives at their slower speeds at the same time. Onboard raid controller could be perfect, so that the pci bandwidth only needs to transfer to the drive controller and the drive controller does the mirror instead of using pci bandwidth to mirror.

16:15 if i remember correctly, when task manager says a disk is at 100%, that just means it was being actively written to 100% of the time since the last tick that it measured.

Thanks SABRENT for making this happen!

Man, I remember back in the day running two of the smaller Raptor drives (32gb?) in Raid 0 and just loving life. I may not have had the best graphics card or processor, but I loaded into BF2 faster than any of my friends for a while. What a time to be alive.

Top tier cooling solution Alex good job :D

First thing I thought of was for video editing. A feature film at 8k raw can take up massive drive space and if you collaborate on a project this would be ideal. At about 8 tb's per hour of raw footage this would be ideal.

12:25 55°C is apparently the sweet spot for nand flash, it doesn't like being colder or hotter Colder = increased data retention, write wear Hotter = increased hardware degradation

You know this just makes sense having cards like this with chips that can handle all of this. I can not imagine how much this could help once pcie 5 really starts to come around in the data center. If they come up with a card to be able to double the amount of drives using pcie 3 with the same card effectively that would keep more out of landfills long term. Love this techy

You guys are doing the kind of thing that I always wanted, but never had the money

Such a great clip! :D Brought back good memories when new hardware was exciting.

We finally found a use for the Radeon VII: A measuring stick

Just imagine how insane the amount of storage you'd have with multiples of these! Its crazy that with only a couple of these and you're already at a petabyte, and in such a tiny amount of space. Crazy.

Nice seeing Microchip being featured in something mainstream. They're a wafer producer here locally. I've done some contracting work for them in years past. They have a large campus in Chandler, and another couple fab plants in Tempe here in Arizona.

13:30. There are brackets you can buy that let you attach pc fans to them to blow on cards in pci slot. They would actuallyvwork really well for this particular set up

Linus: I will suck on your toes for this many drives. Sabrent: Send it! 😂

Clicked on this video so fast that even the SSD couldn't keep up 😅

It sounded quite logical. If the card supports up to a 100 PCI lanes & every M2 NVME slot is wired to 4 lanes, while you can put 21 SSD's on it, it only has 84 lanes out of the 100, for the SSD's & still some lanes left, to the limit of a 100 (just calculated it)... Also if I ever get one of those, I'd think I'd never need another Kb of extra space extra (ofc they wear out after usage, but you get my drift ;) ) Cool vid LTT :)

I just noticed that's an open benchtable. I love my V2, best bench ever. What a great idea.

Those chips are used quite a bit in NAS and storage backups in my enterprise environment. Dell Avamar and Isilon units will connect to switch fabrics with those chips. Very fast stuff!

13:49 there is something so hilariously silly and simple about Linus using cardboard in a cooling solution.

That commercial chiller that you have for your lab should be used with this on a milled cooling block just for this application. See what you could do as far demos go. You might be able to see those numbers climb higher.

Bro could have 66,945,606 (66 million) copies of the original doom

I love these types of videos! Just super cool state-of-the-art technology. Just imagine the future where this type of technology is just standard tech. Keep doing these videos LTT.

I could probaly see a card like that for bio-informatics use case. Wholegenome sequencing datasets can be multiple TB of data, a card like that would 100% speed up analysis of such masive amount of data

Over 10 years ago, before discovering Netflix back when it was good, I would be asked by my family to, find, and host entire seasons of their favorite shows as well as some of their favorite movies. I used to spend ages burning custom DVDs, but we were in the future with a 1080 HDTV and our Xbox 360 could connect to my PC using windows media center turning it into an HD streaming home server on the side. This meant a lot of time downloading overnight, and my 1TB-ish hard drive with more storage than I thought I would ever need was starting to actually visibly fill up, as I also had many multi-gig games as well as rips of the disc images for easily swapping into an emulated drive without swapping/scratching physical media, meaning my physical games needed space for the install and ISO, as well as gigabytes of music in my "My Little Pony Fan Music/Remixes" folder alone. Basically it was a well-loved PC that I volunteered my extra space to turn into a windows media center HD video streaming server for my family on the side. Now Netflix doesn't have all of the good shows anymore, and it is so expensive and inconvenient to be able to stream all of the shows you like as you juggle tons of account credentials, and it is getting harder if not impossible to have an ad-free experience no matter how much money you throw at them. So what if I wanted the entire library of every good show and movie stored locally in HD, or even 4K as we just upgraded a few months back? I don't want TV myself but I doubt the streaming services or our ISP are letting us stream much if any 4K content despite us having the fastest internet package in our area and I purchased a 4k streaming capable router last time we needed one so we could be 4K ready when the time came. Well, acquiring thousands of hours of TV and movies in the best quality available is its own beast, but should I ever win the lottery at least I know there's a product that can theoretically work with a home machine, with a modular nature so extra storage can be added as needed instead of having to buy a larger drive and copy the files from the old one, and also doesn't require that you buy $30k+ worth of SSDs up front and just add more as needed. Basically, if you're willing to spend your time ripping a massive collection of 4K blu-rays and whatever else it takes to get your hands on every episode of every show everyone in your household loves, and the movies too, in the highest quality available, you can make your own media server with no ads or switching between 3 different services to watch every Star Trek series and movie. No "we can't watch this because my parents in the next state are on the account right now" It would be a huge money and time investment to get the card, drives, and all of the media, but it is a lot more technically feasible now to have your own HD/4K media center just by adding an extra card to a regular PC.

8:42 Hot Swapping SSDs feels like Star Trek Isolinear Chip stuff. :D (How they "program"(?) something into the chip & physically plugs it into the star-ship or system it will be used in 20:23 )

I could see this working well for engineering simulations (FEA, CFD) that are effectively limited by the amount of ram that you have. Before something like this, maxing out the ram and letting the solver use the hard drive would increase solve time by something like 10x.

Would be nice to see a scaled down version of this, for those who want this kind of thing but without quite so much cost or overkill.

this is exceptional Thanks Linus

Our R7515 are running - at least similar - chips. There are "only" 128 PCIe-Lanes on the Epyc. But there are 24 U.2 Slots.

I always really love Alex's solutions, they're why I always get excited for a new video with him in it.

Sabrent just asking for a shoutout and casually sending that many 8TB SSD over is a big W move! They truly support Linus craziness and we all benefit from it! :D

Can't wait to see the PCIe 5.0 version!

Yay the dependable cheap prepaid celluar provider I been using for a couple years is now sponsoring one of my favorite channels :D

If you asked me 15 years ago where storage speeds could be I wouldn't have guessed this fast... It's hard for me to be super excited because I feel like the consumer application isn't really there... but the use case in cloud computing will be huge and we'll see it's affects in our services. It's interesting how obfuscated this technology is for general consumers even though we'll all see benefits.

That is insane. As a database engineer I want one of these things.

Use some M.2 to HD MiniSAS adapters and connect the card to U.2 enterprise drives, you could go with gen 3 because of the oversubscription. Use two of them to connect 42 U.2 drives RAID 0

I would love a few cabinets with two of those in my datacenter

I love how they included that clip of shocking hardware to test if it can be killed while Linus was holding it with his thumb on the slot contacts. You just know someone had commented how he was going to zap the board. Edit: oh man, this just reminded me of something I had back in the IBM XT days. Back then I had a 20 mb hard drive built into an ISA card, aka a Hard Card.

Now if only Optane was still around! :D

I want this now in my next build.

12:00 I LOVE THAT BOARD

Nag them for another carrier card, fully load them both, and run ZFS or LVM. Mirror each SSD to its counterpart on the other card, then stripe them. Parallel read IOPS will at least double, if the motherboard and OS can handle it, and you'll be able to lose up to 21 ssds without the array failing, as you'd have to lose an entire mirror (2 sticks on the same stripe) for it to fail.

Having two of those cards, each RAID-0 running as a RAID-1 would be pretty much perfect setup for a big database server. Combine those cards with say 1 TB of RAM and you can execute huge queries very rapidly against a 100 TB database.

Great video, this is very exciting stuff!

Well, io_uring is a very very modern API for asynchronous I/O recently added to Linux 5.1, so it makes sense it'd be able to efficiently take advantage of the incredible performance this board obviously allows to reach. Still though, 44M IOPS is just nuts

The primary use for these PCIe switches are actually in servers - when there's lots of SSDs in a server backplane they're sometimes connected directly, but often there's one or more big PCI-e switches driving the slots. And each switch will have a 16x or 32x (yes, this is a thing even if the standard only lists up to 16x) PCIe 4.0 or 5.0 uplink. This is the real reason why PCIe link speeds are again trending upwards quickly after a long hiatus (between 3.0 and 4.0), there's finally a user (servers) that are willing to pay for development (there's not enough entuast to pay for it by orders of magnitude). And I assume the reason the card doesn't have any fans is because they expect it to end up in servers where there's a large amount of airflow provided by the server itself (sometimes capable of cooling 400W GPU or AI accelerator with just the servers airflow, no fan on the card. It's also unfortunately why PCIe switches disappeared from consumer motherboards (many early SLI motherboards had them), the PCIe switch vendors jacked up the prices by 10x because most were sold to server vendors and they were willing to pay. "Better" (IE expensive) server backplane often accept PCIe (U.2 or U.3, both 4x PCIe), SAS (12Gbps) or SATA (6Gbps) in all slots by routing each slot to either to the PCIe switch(es) or the SAS switch(es). Yes, SAS switches offers all the same features (fabric, multiple servers, dynamic load allocation to the "uplink" and so on) to both SAS and SATA disks, they've been in server backplanes for a long time (long before PCIe SSDs were a thing, never mind M.2). It does makes me wonder if some of the PCIe switch chips also does SAS, that would reduce chipcount and make routing far simpler which means they can ask the server builders for more money...

That would be great to pair with a heavy GPGPU load such as CFD or FEA.

Need to do more testing

These need some sort of Server Conversion where each SSD has it's own Hot Swap Drive Bay instead of needing to have the SSD's installed onto the Cards

honestly if this gets affordable for the average consumer, this could be an absolute game changer for any system and game developers

Probably one of the more interesting videos as of late. Clearly I need to do more research on SSD tech, PCIe, and RAID. This kind of went over my head at points.

I just read about this thing the other day! As someone who's been eyeballin' PCIe M.2 carrier cards, I nerdgasmed 😊

As the SSD capacity fills up, it has to do "garbage collection", which is the process of erasing blocks which have invalid data. The still valid data must be written to free blocks before the erase. Due to high latencies for erase and write, there may be conflicts resulting in requests waiting for GC.

I would LOVE to have one of those in my dev computer. Imagine the gain! Developing 3D games / VR / AR applications can take up a LOT of storage space. And compiling / rendering needs to read / write as much data as the CPU / GPU can handle. Add project management / version control, and both capacity and speed of a card like this become VERY interesting indeed! Of course I can't afford even a basic dev system (not even 4.5K euro), let alone add this card XD

Linus's face when Alex mentioned chia mining 😂😂😂

9:30 you should do some custom cooling for it, test it with water cooling and other solutions lol see it if affects performance under a heavy load of whatever kinds you could think of

If I were to have like a private Emby media server with a bunch of these ssd cards I could fit thousands of full shows. And I mean all episodes of all those shows, from start to finish, and still have storage for all of my games. Sheesh that's a lot!

Great video, would love to see more affordable non-Bifurcation options like the highpoint for comparison. Also thanks for the linux testing as well! Some of us don't use windows 🤣

The AI thing does make sense especially when combined with DirectStorage-type tech.

100 PCIe lanes on the PLX chip is divided up as: 16 lanes for the slot + 21 * 4 lanes for drives, 16+84=100 lanes!

as an audio engineer w an electronics background, this is impressive!

Thanks to you Linus, I'm building my first PC this week!

The Radeon VII ngl was such a nice looking card. I vote for Radeon 8000 to adopt a gunmetal box design.

For CGI simulations this is amazing, Right now we have to load assets and all data on RAM so the server ends up having tbs of ram and that is also very expensive and not as flexible in size like this

you know it's fast when Blu-Rays/second is a legit unit of measurement for the bandwidth