What a day to be alive! 🤯

I remember writing 2 years ago that in the next 10 years we'll have video generation and text/script generation so good it could bring back cancelled series. Didn't expect the possibility to come so soon. There's barely anything holding it back anymore

0:40 tram casually crushing a dozen pedestrians

"Can you clean the dishes?" "Yeah just a moment, I'll clean them just two more papers down the line"

I follow Karoly since 2018 . And now I know that his favorite paper is that latent space paper. If he talks about an simple subject like an hamburger, he will find an way to include the latent space paper to talk about it

It's occurred to me that history is repeating in many cases. Photography came along and felt new and strange in the latter 19th century. Then cinema arrived and would’ve shocked people in the 1900s/1910s. And now generative AI has emerged in the early 2020s and is shocking the current world. We're just a part of history really, aren't we? I think it's easy to forget that we're part of the same timeline, even if history feels like something completely separate, especially when going 100+ years back in time.

I imagine that you could create a storyboard of key images, with or without sora, and use sora to interpolate between them in a meaningful manner. A form of direction for longer videos.

I'm studying AI (graduating in a couple months) at university and have been watching this channel for a while now. I often mention your videos and share them with my professors or other students. I shared your enthusiasm, what a time to be alive indeed!

6:50 this is similar to how the human eyeball has a blind spot, but merges the differences of the many to cover the blind/fill it in. The same as how many humans and the differences share and flow the energy between to bring out something different like a movie or a new universe through a simulation that is a matrix of differences that reflect reality and can find things we haven’t uncovered yet.

I think most people don't realize the most incredible part of this technology. If it can make life like Video then it has the potential to make life like Virtual worlds. As in you can explore a completely life like Virtual worlds using a VR headset like the apple vision pro pro.

Making special effects for movies will be so easy that literally a kid with no prior knowledge could do, it is amazing and i hope i can download and use this soon

"Hey Netflix, play a sci-fi movie set in the not so distant future, with a synth soundtrack, in the direction style of Darren Aronofsky, and I want the movie to run for 120minutes"

Your passion and teachings are inspiring! Thank you for the amazing content, always :)

Thank you, Dr for going a bit in depth about how this technology works! We scholars love the details

The real kicker is not that it creates videos but that it´s a huge step forward world understanding like physics and how objects in the real world interact with each other. That´s the holy grail of AI right there. Just like it "understands" fluids in a way that´s a whole subsection in graphics and mathematics without using them is mind blowing. It´s not perfect right now but it´s crazy we are seeing this, something that was unheard off like 2-3 years ago with experts saying we´re a half or even a full generation away from that.

Real-world physics simulator has wide-ranging implications that we've just begun to unravel! What a time to be alive!

Impressive, what a time to be alive!

I've seen other videos on SORA but this video gave me an important insight by comparing to GPT2. Now imagine how good this would be and the emerging properties it'll have when the GPT4 equivalent comes out. Probably could make a whole movie.

Never stopped to think about the fact that a model capable of creating convincing video would, of course, be able to make almost flawless images. When these systems hit the retail market, everything regarding marketing and concepting is going to change.

After reading the paper, the paper doesn't refer to what the Unit of Compute is for the comparison of base compute, compute x4 and compute x16. I feel like that should be defined before we can really say how much compute is necessary to create a good video because if compute x1 is a unit that is beyond the level of anything that requires a cloud solution I think that there's quite a bit of work to be done before we can see Sora being used in a commercial or retail use case. Also another thing that needs to be revealed is how much training time was used and how long does it take for Sora to produce results like these. But it is indeed exciting to see the technology growing. I just hope that OpenAI would release a more detailed writeup soon.

I'm excited to try this once it becomes available. I'm really interested to see how the new and improved image generator will work as well. Dall-E 3 is already really good so it should be interesting to see what it can do.

I find it hard to be optimistic about Sora Ai. It's an incredible AI where I can't even begin to fathom the hard work it took to make, but I can't see how it'll impact the world in a positive way.

using video generators as image generators makes so much sense because pictures are nothing but moments of the non-stop video we call life. so getting an ai to do the same process as us andnot only the end result will definitely improve image generation specially the photorealistic results

Love Sora, Cant wait to add it to my autocoding program to make video tutorials too!

It’s already a superhuman level of visual intelligence in some aspect. It can do fairly reasonable, physically plausible predictions in this universe. Now I’m wondering how it would be after one more revolution. Sora feels like Alpha Go Lee. How it will looks like after it becomes like Alpha Go Zero. Entire film and game industry including XR will change forever.

As much as I love the work that goes on in this field. This is had gone on long enough

That reverse mode could be really cool for innovative movie scenes

Unbelievable 🤯🔥

It's such a huge step forward, it's like going directly from GPT-2 to GPT-4 ! o_O

can't wait to be running such a model on a consumer GPU in the near future

1:53 man got six figures

All I want is for Sora to make me a new season of the old red dwarf with the original characters!

The cloud man is awesome

Can't wait to see the video about Stable Diffusion 3. "Two papers down the line" seems to be flying past every two weeks at this rate.

3:32 - OMG, this is a new one for me with this lady with the glasses and I really thought at first that she's a real actual person in a video so got confused for a sec as to why would you show a real person like that :) so crazy that it can even still fool even those that already know Sora, goes to show how totally convincing already their videos can be (at least for a first glance at it)

We're very close to being able to see actual memories and images from our mind.

Regarding temporal coherence, it’s actually quite straightforward. The concept is that an image can be represented as a 2D grid, while a video combines a 2D grid with an additional time dimension, effectively creating a 3D grid. The AI is trained to generate a single 3D object, and what we see at any given moment are slices of that object.

7:00 This picture in the SORA technical report was misleading because the denoising happens in the latent space, not in the pixel level

What a life to be a time!

its learning physics by looking at videos, thats insane!!!

Very impressive, they took an already proven solution for still images and tweaked it by adding another dimension. I can see potential for this being used in many other fields not just creating videos. Perhaps one concern will be compute power as it scales up?

I want SoreTube, showing content creators from a alien race somewhere in the universe. 24/7

I was listening to Prof Stefano Ermon at Stanford talk about Sora, and was blown away by the idea that a diffusion based transformer model could 'understand' physics without a physics simulation model of the type used in game development (eg Unreal Engine). A recent article on Semafor revealed that there is a startup company that using the same type of modelling to predict the weather, and does it better than NASA, the military and any current weather prediction model... and has achieved this trick by associating real-world weather balloon data to previous word-based weather predictions. Again, no physics model required! This weather forecasting system is now being adopted as it only needs training once, and then can run very efficiently - more so than traditional weather system models that try to simulate the actual physics, and require enormous compute power. It leaves me wondering how far this approach can go... I still don't believe it'll arrive at any sort of real world agency, let alone consciousness. But tools like these will not need human qualities to change the world. As amazing as it is shocking: the pace of development is way faster than society is ready for.

The videos generated are stunning visually, but it looks like watching in slow rewind mode ! 😵‍💫 which makes it land close to the uncanny Valley 😵‍💫

What a time :)

The wildest thing to me is how it interprets physics

Sora's videos seems like dreams...

The fact it wasn’t open for use speaks volumes of how insane the cost must be even just inference

Is the reason that the latent space is used, that the latent space is closer to the output, so one can fully utilize the distribution of (latent given output). Rather than (input given output) as the latter is very far. Then translate to both modalities, the latnet can be editted to perform "better" ?

Very cool

Do we know the amount of compute power used for generating these videos?

It’s like what it’s like seeing into a dream

Possible best use case is using Sora to create movie company intros

That video from 6 years ago is a shocker, its so far away from where we ended up now that I am almost frightened to think where we might be by 2030... I mean all this has to be extended to 3 dimensional space (AR/VR) in real time with real interactable characters on demand so basically you are looking at The Holodeck...

🤘

2:00 is that a pizza tower?

So when will this be available to the public? Because I want to use it! I've got stories to tell, Series to write!

When Nvidia releases their 100GB VRAM TPUs we will an even bigger jump in speed of progress of AI development.

That technique that gpt2 used to read multiple reviews, could we use that technique to have it read papers on computing and have it theorise improvements? Would it be viable?

Imagine having this turn your favourite book into a movie... With a few twists just for you... And whoever you want to see in the lead role...

This guy really loves explaining latent space

0:38 I would not consider a magical sky tram coming down onto a piece of land supported by the Golden Gate Bridge to be "completely natural".

We are getting much much closer to really implementing The Entertainment :D

You are wrong this time. SORA takes patches of temporal and spatial data from latent space, not pixel space. Love your channel always. Thx

Hold on, if you can generate a video out of a still image, does that imply that you can generate another 1 minute video out of the last frame of the previously AI generated video? If yes, it's possible to extend the video exponentially and create scenes for a TV show with this trick.

The patch encoding seems like it could be used as a video compression codec

i wanna try it with old pictures ive taken!

Sora! Sora! Sora!

That's all very cool, but I'm way more excited for Stable Diffusion 3

Clip# 2 [ 2:45-4:24 ] SORA is denominated in 'patches' rather than tokens

2025 will be the year of robots That can do all we want

day after day I believe more and more in the theory of simulation reality haha

imagine when AI will create reels on Instagram that would increase, thou screen time to sinister level🤯😵‍💫 it would never let your eye off the screen

In the next 20 years computers will be able to make food appear from prompts that we can eat. When I was a child I used to hear stories that mankind long ago could raise their hands to the cloud and bring food out of it. Sounds similar.

Imagine this with sound generation ... Only a few months from now

My prediction is we will have simulated computer games from prompts within....3 years, and they will look and play better than current games today, and they will be catered to our own specific preferences....wow, what a time to be alive!

can you imagine this with in-game engine overlayed with polygons so that you can actually control the game?

Is it also possible to enter a start and ending frame?

"you can count on me, being here" Seeing the speed of this development, i am not too sure about that. Either because it all goes wrong or maybe you have your personal agent doing these vids for you then.

6:35 es de la era del hielo 1 esa escena de los dodos

Just imagine real-time AI video gen but you can control a character like you would a third/first-person game. Games would never be the same

we need to democratize this tech more; while acknowledging the potential for misuse of powerful technologies like SORA, it's important that these technologies remain accessible to well-intentioned individuals. This approach would allow for positive use and innovation, akin to how regulated substances like alcohol are managed, balancing the risks and benefits.

Countdown to full dynamical 3d model output: 3… 2… 1….

AI learning something completely unrelated to its task because it is something that happens to help out performing that task? Where could this possibly go wrong? All jokes aside, what a time to be alive.

can sora be used to finally make a non-janky looking higher framerate video from lower framerates?

One day, we will see conventional computing like we see black and white tv's

Neural rendering is the future

When is it coming out anyway?

So, could I eventually input my favorite novel, give the AI vague or extremely vivid details of cinematography, character development, musical scores and compositions etc, etc, forgive me I do not have the film vocabulary….but you could request it to be directed and produced in different styles related to eras and actual director and producers styles in an almost infinite array and hit the “play” button over and over again until “you” create “your” masterpiece!?! How far away are we from that? Just a regular ole Joe like myself creating theatrical masterpieces??

Martin Scorsese: Marvel is not cinema Martin Scorsese: Sora... *sigh* 🤷‍♂

The more computation you do the better, time to accelerate quantum computing technology advancement

im really curious about how much compute power it requires to generate these videos. i have a feeling that it requires a super computer otherwise they would have mentioned how little it takes to generate.

Jezus if AI progress continue any further this guy will cream his pants.

I feel like we're slowly figuring out how to create the matrix lmao..

I work with Local Businesses & a big pain point is getting images. If only I could put an image of one of my tradesman within a video ….

🎯 Key Takeaways for quick navigation: 00:00 *🚀 Sora, OpenAI's text to video AI, can create high-quality videos from text prompts and extend still images into video, both forwards and backwards.* 00:24 *🔄 It can also generate several natural-feeling scenarios leading to a prescribed video ending.* 00:50 *🌟 As a researcher in light transport simulation, the speaker expresses delight in Sora's long-term coherence and the quality of its reflections and refractions.* 01:19 *📈 Sora can produce videos up to 60 seconds long, showcasing advanced details like dust and grease marks.* 01:45 *🎬 Sora is capable of synthesizing a complete scene in one go, without needing to splice various parts together.* 02:06 *∞ One impressive feature is the ability to create infinitely looping videos, alongside limited physics simulations.* 02:35 *🎨 It can generate highly detailed still images at a resolution of 2048x2048, surpassing the capabilities of DALL-E 3.* 03:26 *🧠 Unlike large language models, Sora deals with visual content in a unique way through 'patches', allowing for effective video generation.* 04:21 *🌐 Sora operates within a 'latent space' that allows for the generation of new, similar materials or visuals by navigating through this space.* 05:11 *🤖 By understanding the structure of English language and sentiment detection, Sora can accurately extend images into coherent and physically accurate simulations.* 06:09 *🌌 A diffusion-based transformer model Sora to avoid flickering effects in videos by refining multiple 'bunches' of noise simultaneously, considering the entirety of each image in a sequence.* 07:26 *💡 The rapid advancements in AI video generation exemplify the power of human ingenuity and computational research, highlighting the potential for future breakthroughs.* Made with HARPA AI

The real question I have is where is that lightning bolt coming out of the angry cloud guy? Looks painful.

Another channel said that it would take 24 hours to produce 60 seconds of video. It's still a remarkable accomplishment but if it takes that long to create video there will be a lot of obstacles to using it for replacing current video production.

The 1x compute dog in the snow is kinda scary...

Great AI