How to Convert ANY 2D Video to 3D Spatial Video on Vision Pro with AI ft. OpenAI Sora

День тому

Curious about turning AI-generated videos like the latest OpenAI Sora into captivating 3D spatial experiences inside Apple Vision Pro? Look no further! Join us in this tutorial as we explore the fascinating process of converting any 2D video into a mesmerizing spatial video on Vision Pro using AI. Let's learn these AI transformation tips step by step!
🥽 Download & Watch the final result in Vision Pro here: • OpenAI Sora to Apple V...
0:00 - OpenAI Sora vs. Vision Pro Spatial Memories
1:12 - OpenAI Sora in 20 seconds
1:38 - How to Turn 2D video into 3D Spatial Video
1:52 - HaloDepth 2D to Spatial Video
2:51 - 2D to 3D video the FREE solution
3:46 - Owl3D best settings for 3D conversion with AI
4:59 - 3D video to Apple Spatial Video using FREE FFMPEG
5:13 - Use Handbrake to fix problems
5:30 - My automation code for batch processing Spatial Video
6:09 - 2D Video to Looking Glass in 3D Hologram - workflow
➡️My laptop for AI: aorus.shop/HughHou.
➡️OpenAI SORA: openai.com/sora
➡️Detailed tutorial on 3D to Spatial Video is here: • Master Spatial Video: ...
➡️FREE FFMPEG Tool: blog.mikeswanson.com/spatial
✅Copy and paste my FFMPEG automation code here: bit.ly/48lFHEk
➡️Handbrake Video Converter: handbrake.fr/
➡️HaloDepth 2D to Spatial Video Converter: halodepth.com/
➡️Owl3D: www.owl3d.com/
✅10% discount code for Owl3D: HUGH10
🐕 FOLLOW ME & Kimchi:
➜ Instagram: / hugh.hou
➜ Facebook: / 360creator
➜ Meta Quest TV: ocul.us/30uMZUj
➜ TikTok: / hughhou
🤝 We are open for HIRE! Reach out: hi[at]hughhou.com
#visionpro #openaisora #2dto3d

КОМЕНТАРІ: 117

@hughhou 2 місяці тому

So Owl3D switched to subscription now. But I still have good news for you. Here is a 10% off code you can use: " HUGH10 " on any plan. This is NOT an affiliated code or link (I don't get anything - 100% giving back to our viewers). I feel really bad for the misinformation, so I hope this makes it up to you. As always, I am doing my best to serve the community. A thumbs up and share are greatly appreciated!

@dtaylor7798 2 місяці тому

Wow, it would be interesting to compare a 3D video shot with a VR camera and a video shot with a regular camera and converted to 3D using software. If the quality is similar, then this will create an explosive growth in content for VR

@jakemathew 2 місяці тому

The 3D conversion quality of Owl3D is amazing. Many 3D videos on UKposts and DeoVR were converted using Owl3D.

@hughhou 2 місяці тому

Sadly, right now there is still a big gap. Shooting 3D is still 10000x time better than GenAI - but the gap is closer everyday.

@paulct91 2 місяці тому

....it can't be "that" good yet, even OWL3D is kinda meh, works to make 2D to SBS (VR180), RGBD (LookingGlassFactory) Hologram Depthmaps, and classic 3D Red & Blue picture/video conversions, I'm referrring to OWL3D.

@jakemathew 2 місяці тому

Owl3D definitely does NOT convert 2D to VR180. Not sure where you heard that.@@paulct91

@bobbyDig 2 місяці тому

It's not so much about the camera, but how you frame your shot for Spatial video. I got some great results with 2D video conversion the HaloDepth.

@Gorto68 2 місяці тому

Hugh -- here to show my support and learn from your explorations. Excited to watch this stream and share to my circle.

@hughhou 2 місяці тому

Awww thank you thank you! Good to see you live!

@charleslorne 2 місяці тому

Dude you are THE most knowledgeable and resourceful person on this topic on the entire web period!!

@hughhou 2 місяці тому

Awww thank you I am so glad you think so :)

@ElectricVehicleSpace 2 місяці тому

How do you record the view from inside the camera?

@hughhou 2 місяці тому

You can use Xcode. Or just the native recorder under Control Panel.

@criticalkids 2 місяці тому

Love your videos man!

@hughhou 2 місяці тому

I appreciate that!

@DillonThomasDigital 2 місяці тому

Great video! Glad to watch it as it premiered! Question about the export to looking glass hologram - this obviously spits out a more than stereoscopic image - how would one export something for viewing in quest3? I've been anticipating the democratization of volumetric video, and it appears we're encroaching on that now. Thanks again for your work!

@hughhou 2 місяці тому

So looking glass is basically a RGB video with depth data (depth estimation). It has focus elements to make depth look better in Looking Glass or any hologram display. In Meta or any VR headset require higher resolution depth map to do Volume / Volumetric. There are several papers in active developement on this topic and none of them is good enough for pratical use yet. Kinda like early day of NeRF. But thing will change anyday as it is just science lol

@Billwzw 2 місяці тому

How does the 3D conversion quality and artifacts compare to Leia Lume Pad?

@hughhou 2 місяці тому

Leia Lume Pad is also really good but I don't think they open source to Vision Pro tho, right? I will do the comparison in the next video. As Lume Pad is an entire different breast lol. In fact, may way to approach this topic. I should make a 20 min video instead of a 7 min one lol.

@writeforanimation 2 місяці тому

Do these videos have the same resolution limitation from Apple, or can you play at 4K resolution from the Apple Vision Pro?

@hughhou 2 місяці тому

What resolution limitation? The limitation is only the M2 chip and as long as it run. I can play 4K per eye on Vision Pro and even pushing 5K per eye (not displaying it but serving it).

@writeforanimation 2 місяці тому

@u When you take Spatial videos with the iPhone 15 Pro, they are limited to 1080p.

@MrLiree 2 місяці тому

At 3:13 ... Where did you get your information about the pricing of owl3d? The software states a monthly subscription. Or did I misunterstand sth. here?

@hughhou 2 місяці тому

B/c I bought it like 6 - 8 months ago. It was 1 time fee but I guess they are launching something news let me circle back on this. Cheers!

@anivedk 2 місяці тому

Hi Hugh! Do can we do this on a mac m3 ? Which apple model do you suggest to make content for meta quest 3

@hughhou 2 місяці тому

I use M3 Max but any Apple computer should work. PC is actually better in AI still.

@anivedk 2 місяці тому

Hi Hugh! How do I make the video pass through , do I need a green screen? Not clear

@hughhou 2 місяці тому

Pass tho need Green Screen.

@taosranch 2 місяці тому

Thanks! Another great video. But any chance of Owl3D or a like competitor coming soon to the mac? I'd prefer to keep my workflow all on my m3 macbook pro.

@hughhou 2 місяці тому

Yes. HaloDepth is making a desktop service for Mac. And Owl3D is making something called Studio as well for any computers. So everything on this video are on active development - so workflow can change in coupled months. I will make update whenever those are available :)

@jakemathew 2 місяці тому

Owl3D released a Beta for their Mac app today.

@TV355 2 місяці тому

@@hughhou HaloDepth now offers free conversion up to 3 minutes in Full HD.

@jamesburland 2 місяці тому

Superb! Thank you.

@hughhou 2 місяці тому

Our pleasure!

@knmisra 2 місяці тому

Looking glass vs lume pad 2. Which is a better technology/ product overall?

@kingnt66 2 місяці тому

Thank you Hugh! You are always on top of the game! How does the quality compare OWL3D with HaloDepth? I use Owl3D but never use HaloDepth

@hughhou 2 місяці тому

Owl3D right now still better I think but HaloDepth has update like almost every days so it will prob pass it soon :)

@jesusmtz5141 2 місяці тому

General question, i am trying to convert 3d hsbs movies to spatial video using ffmpeg with your code, but all i am getting is only one side and it is misplaced, any suggestions?

@FlexScreen 2 місяці тому

Hey Hugh, love the tutorials. Where can we find the lifetime license for Owl3d?

@hughhou 2 місяці тому

So apparently I am wrong. Lifetime license (the one I got) is grandfather account. Since now the stable release only have subscription. But I got a discount code for you guys in the pinned comments. Might end up be cheaper.

@urbanvr 2 місяці тому

Hugh you're a genius 🙏🏾 thank you, for what you do.

@spediegunz 2 місяці тому

have you compared the output of owl3ds 2d to 3d conversion to that of using apples built in spatial recording natively using iphone 15 does AI 3D look better than the real thing?

@jakemathew 2 місяці тому

The iPhone 15 Pro can only record spatial videos in 1080p at 30fps. Owl3D can create spatial videos up to 8K and 60fps. If you know how to use the Owl3D settings quality of the 3D is just as good if not better than the iPhone 15 Pros

@hughhou 2 місяці тому

You can actually see it from this video side by side. The talking head at the beginning is shot on iPhone 15 Pro Max and all the AI video is Owl3D (well some of them are HaloDepth), and they are intercut together fine. Just like the 2D world, AI is coming hot and fast in all video industry. Here is the video if you are curious (using Vision Pro to exam go without saying): ukposts.info/have/v-deo/qYGnaq-qnYJ2y6s.htmlsi=ehjdhMAnjl9cGtLs

@CarlitoTries 2 місяці тому

There is no 1 time purchase for that software.

@hughhou 2 місяці тому

Oh good to know. Let me follow up with the developer to see what is going on.

@lixinzhao6987 2 місяці тому

这套369眼镜真好看。喜欢！❤

@bujin5455 2 місяці тому

2:58. Wouldn't this work with a 3090? Other than the 4090 being faster, they have basically the same feature set, and the same amount of vram.

@hughhou 2 місяці тому

4090 is just faster - it work with even just your GPU - but really REALLY slow.

@md.fahimalam7481 2 місяці тому

will it be possible with any 2d videos or only AI generated videos?

@jakemathew 2 місяці тому

Owl3D converts any 2D video to stereo-3D. Not sure why OpenAI Sora is in the title of this video (besides to get clicks)

@tshuxeng5267 2 місяці тому

no longer see the 139 buy option. is this similar to ani3d, thier price is only $40 one time fee?

@hughhou 2 місяці тому

They cancel the 1-time fee initial offer :(

@ksnax 2 місяці тому

Almost didn't watch this one as it seemed Apple exclusive. Glad I did though. Owl3D was a tool I did not know about! I just get to skip the spatial video conversion part to work with my Quest 3.

@hughhou 2 місяці тому

Hope you enjoy it! Yes it work on any 2D to 3D conversion but AVP make it do it allover again as the 3D look great there. But if you only have Quest, the Quest Spatial Video also benefit greatly for this approach.

@_Clivey 5 днів тому

03:39 Where is the 2D to VR video you said was coming next? I cant find it?

@FriedguyFriedguy 2 місяці тому

How can I use a Pc to transform my owl3-d creation 180vr video to spacial video? I only have a pc no Mac.

@benhuson1 2 місяці тому

I simply transferred the video to the headset and it plays in 3D SBS on the headset (EDIT - this was on the Quest 3, just to confirm)

@hughhou 2 місяці тому

Do you have an iPhone? If so, you can use Spatialfy or a almost free web service by SpatialGen (the company made HaloDepth). See the tutorial here: ukposts.info/have/v-deo/iHJ4g6aLqoWqsHU.htmlsi=2UduGyUUDYnSlHj5 (almost at the end after the Resolve workflow)

@Bil30321 Місяць тому

Can you make a 180 3D VR video of a Star Wars X Wing fighter going through Death Star trenches using Sora?

@hughhou Місяць тому

Yes! I am planning

@Bil30321 Місяць тому

Here is an example of what 180 VR Star Wars could look like, although this was made in Blender, not Sora, which will be way better: ukposts.info/have/v-deo/n6dldauLaYZ1lGg.htmlsi=CrwcGQb4lRRqFy_d

@Archway3D 2 місяці тому

Hi, Hugh! Thanks for making this great video and I will pass it on to Leon Lu who is the CEO of Owl3D. He will be very pleased. By the way, I recommend "render right side only" as render mode. This way, only the left side is converted with the depth map, and the right side is the original image, so you don't feel the distortion so much. Actually, I taught him this!

@hughhou 2 місяці тому

Where is the "render right side only" options? And why this is better? Sorry I just want a little bit more technical insight.

@Archway3D 2 місяці тому

@@hughhou From "Advanced setting" after you select video to convert on Owl3D, you'll find Output render mode(by default "Render both sides"). Then chose "Render right side only". The converted video image may have the distortion in case of the scene foreground and the background were complexed. When the both side of left and right have each distortion, it's hard to watch the scene as a stereoscopic. So one side original and the other side converted are so efficient to watch the half of the scene no distortion. I prefer "Video 3D effect" to set Strength 9 because the one side is not converted. 75% people uses right eye mainly so that's why I recommend this way I don't feel stress.😁

@leandrogoethals6599 2 місяці тому

yo guys do u know any open-source scripts where u can use a depth model to sbs?

@hughhou 2 місяці тому

Yes here: blog.mikeswanson.com/spatial

2 місяці тому

That's great. ❤

@hughhou 2 місяці тому

Thanks! 😊

@toralfczerniak8250 2 місяці тому

Does anyone know if I can run it on two devices at the same time if I have a license? Converting takes a while. It would be good if I could run it on two devices at the same time and convert two videos at the same time.

@hughhou 2 місяці тому

Yes you can I believe - but I will invest in a powerful GPU instead of invest in 2 computers. GPU with high vram make all the different.

@DSK123 2 місяці тому

I guess I’ll have to wait and see when the video premiers but I’m left wondering if this is true spatial (in a previous video you mentioned parallax) or “just” 3D. I’ve been playing around with 2D to 3D conversion for some time now and it’s awesome but I’m curious about Spatial.

@Instant_Nerf 2 місяці тому

To me Spacial means layers of objects , between them .. space between space, you can go between.. What I see on Vision Pro is depth not Spacial. I really believed they were using the LiDAR to create these Spacial videos in the beginning 🤦 there is a hologram app on the Vision Pro that does it cool! But it’s a meeting app. It takes the depth camera on iPhone creates a 3D persona and imports that into the app as a hologram .. you can actually walk around the person.. pretty wild

@joelface 2 місяці тому

@@Instant_Nerf That's what I was hoping as well and was feeling pretty disappointed about it. However, the interesting thing about Apple's spatial videos is that they're planted in space and when you move you can see the scene from different angles (I believe). Almost as though you're looking through a window at the scene. With a regular 3D movie it stays in the same perspective no matter how you move.

@Instant_Nerf 2 місяці тому

@@joelface you can look into it like a window but the angle doesn’t change .. I know I have one, the other mode you can clearly see the whole image move, when you expand the Spacial video full screen.. the whole 3D image moves .. causes a little nausea for me. It’s like stereoscopic with added effects. What I was thinking the same way they scan and create a persona of you.. it would be wild to scan an environment and 3D reconstruct the environment, we can walk around in.

@joelface 2 місяці тому

@@Instant_Nerf thanks for clarifying. I don’t own one. I am hoping that your idea is what we see become available soon. It’s absolutely beyond frustrating that they didn’t use LiDAR at all for spatial video. Seems like such an obvious choice.

@joelface 2 місяці тому

@@stephenbard1590 very disappointing to learn. I hope this is just step 1, and we’ll see a much much larger focus on true spatial video at this year’s event.

@toralfczerniak8250 2 місяці тому

I can't find a Lifetime License there. When upgrading, only monthly subscriptions can be found

@toralfczerniak8250 2 місяці тому

I mean Owl3D

@toralfczerniak8250 2 місяці тому

I would have bought the program straight away for a fixed price. But I won't take out subscriptions for ANY program.

@christofthuis1834 2 місяці тому

My thoughts exactly. We need a competitor with fixed price

@hughhou 2 місяці тому

Hey so when I bought the software, it was fixed price. But I think they change it recently (I purchase it a while back). Let me ask what is other options out there and report back. Sorry for the confusion.

@toralfczerniak8250 2 місяці тому

I heard that looking Glass Factory is behind olw3d. I backed the looking glass portrait and also the looking glass go. Maybe it is possible for looking glass supporters to get special conditions

@VRainier 2 місяці тому

I cant find any option for one time purchase on Owl3D, maybe they removed it? ):

@hughhou 2 місяці тому

That I don't know. I own Owl3D for a while now. Let me ask and circle back.

@toralfczerniak8250 2 місяці тому

same at me@@hughhou

@christofthuis1834 2 місяці тому

I asked. He doesn't. We need a competitor who does

@VRainier 2 місяці тому

@@hughhou what did they said to you?

@seyittahirozuolmez1026 2 місяці тому

Great video. But i surely know that the AI can easily make precision depth map when its in creation phase. all other solutions have a problematic algoritms to generate eye-strain free stereoscopic images just like real-time 2D- 3D TV signal conversion.

@hughhou 2 місяці тому

Good to know. Yes indeed. I hope OpenAI Sora can add this into native generation but not enough 3D media samples out there for AI to learn what is going on. That is the main problem. Sample size is small.

@ted4687 2 місяці тому

Great🎉🎉🎉

@hughhou 2 місяці тому

Thanks 🤗

@brettcameratraveler 2 місяці тому

Midas AI was used last year to convert 2D video to 3D. Curious if this method is better quality. Might be awhile before we can convert 2D 360 and 180 video to 3D using AI as it might require very large amounts of training data from cameras like that. Hopefully they

@hughhou 2 місяці тому

Yes Midas AI I think it is still being used and also there is TikTok Depth anything. I think after this tutorial, I will have several other approaches thanks to our very knowledgeable community suggestions. To be honest, reading the comment in this video is better than watching the video iteself lol.

@christofthuis1834 2 місяці тому

Let's go

@artenman 2 місяці тому

Can I get #2 Pho please thanks, and some chicken spring rolls.

@XAirForce 2 місяці тому

I don’t really care because I never get access to any of the early features even when I have a subscription.

@hughhou 2 місяці тому

On openAI or Vision Pro? Vision Pro has no subscription.

@XAirForce 2 місяці тому

@@hughhou Open AI. Apple doesn’t have to have a subscription because their vendors all do, and then they collect 30% of it. Pretty good scam if you ask me. I went to open AI yesterday to see if Sora was available and I didn’t have access.

@XAirForce 2 місяці тому

Just like I leave comments and they get deleted. I just responded the second ago in the comments gone already. I’m not gonna even bother with any of you because it’s just not worth it.

@JairRiveraFotografo 2 місяці тому

@@hughhou ignore the guy, looking at his channel, he´s not ok.

@ElaztikaVR 2 місяці тому

Great video packed with knowledge 📕! Also, I'm excited to know about the new air or slim glasses by Apple and price I believe 1000 can you let us know 👌also how to upload to Meta TV vr180 & 360 video?

@JohnNy-ni9np 2 місяці тому

There is no such thing as using OpenAI Sora to convert 2D to 3D for now. Click bate and fake.

@brettcameratraveler 2 місяці тому

Watch the video. Sora is just one step - an optional one at that. The end result is yes, you can choose to take a video that was originally created with Sora and process it with other apps to make it 3D.

@jakemathew 2 місяці тому

Owl3D is a great app that I've been using for months and it should have gotten it's own video. Choosing to make a video (and video thumbnail) combining it with OpenAI Sora and Apple Vision Pro just confuses some viewers. Owl3D has existed long before Sora and AVP - they are just the latest buzz words UKpostsrs like to use.