Although I have no hands-on of the AVP, but I can see it's a parallax effect post-proseed on by the AVP spatial video player, between the player window border and the content video. Many others, including myself, were expecting a parallax between the close-by object and the background of the video itself. This did not happen since in core the technology is stereo vision 3D video.

Can we upload spatial videos made using the Vision Pro to UKposts? How can we watch them?

It is 3D video! They are using software and the cutout window to move the video around (rotating it, zooming in and out etc) in space giving an illusion of looking through a window. Most 3D video out there is static attached to your face. If you move your head the video moves with it. However, you see the full video so distortions would be obvious if they were to zoom in or rotate the video potentially making you dizzy. You can achieve the same effect with a 3D video in any game engine or 3D Video player that allows you to move the window, zoom or crop. They are just doing it automatically.

Love your videos, and I always knew you would bring the very best content about AVP.

Hi Hugh, is it possible for you to share a spatial video captured with the Vision Pro in 3D here on UKposts because I'm interested to see the difference in depth and quality compared to the iPhone 15 Pro Max's spatial video (I assume the Vision Pro has a wider stereo base, but I'm not sure about the video quality). Also I was wondering if the Vision Pro has the same format for capturing spatial videos limited to 1080p/30FPS like the iPhone or does it support better formats like 60FPS?

Great video as always! Basic question but how do you play traditional SBS (not VR180)? Does it require some metadata to show in stereo vs SBS 2D?

When on a smaller budget, would the limited resolution footage of the Kandao Qoocam Ego still look ok on the headset?

Would you be able to test what the max resolution Vision Pro is able to display? I have several 10K VR180 photos from a GH5 rig which look good on Vision Pro, but was wondering if it would be worth upgrading to a dual Fujifilm GFX 100s rig or similar cameras for 16K or 24K VR180 photos. Also is 8K the limit for video playback? I tried playing the 12K FM DUO sample video you linked in one of your previous videos, but I couldn't get it to play on the Vision Pro.

So is the spatial video format actually storing extra stuff, or does it just put the viewer in a different mode that has some tricks added at the edge of the view box?

Love your videos Hugh, but in this case am I missing something or is it just depth map from stereo giving that Facebook like parallax? If it is... It's underwhelming and not 'real' parallax (missing information /pixels) being created. Reminds me of the early days of "rubberstamping" 2d to 3d conversions as done on films like clash of the titans. Can't tell much from your video above, but would be good to know if this is the case.

Great content as always! One correction: PSVR2 already has foveated rendering (with eye tracking) and it’s a cheap device. Still having 7 VR headsets at home I can’t wait to try the Vision Pro once it’s introduced in Europe. :)

I've been taking a lot of VR180 pictures with my Lenovo Mirage VR camera. I would love to transform them into the new Apple Spatial Pictures. Any idea how?

Can you tell me why resolution is low when I stream the Xbox app? Everything is crisp until I get into the gameplay and the it degrades. 😢

Hugh, do you have any converted Kandao EGO footage for download and review in AVP?

So the claim is that the spatial video is just 3d video inside a shadowbox, and not actually the slightly volumetric video we were shown in advertisements. It's hard to tell from your example if the actual video has more than just stereoscopic depth. One test would be whether or not you can tilt your head 90 degrees while watching and still see depth. The shadowbox effect looks pretty cool, but I see the complaint. True spatial video is something that exists on Leia's Lumepad. The limitation is the screen viewing angles and the quality of AI-driven depthmaps.

Can one use Skybox VR video player to view spatial videos on the AVP?

Is there a difference between vision pro spatial video and quest 3 Passthrough recording? If so what's the difference?

I am a developer and have Xcode. So How do I add the code to achieve spatial video as you described?

It’s spatial video capture but 2000 x 2000 (square). My family videos are wide screen. Deal breaker?

can't believe you with all your experience is falling for this. I obviously can't see 3D on a 2D video, but I don't see any parallax going on in the image itself, I can't see more to the side of geometry. It was only later that I realized you're speaking about the very cheap trick or only showing a part of the whole picture in frame and then allowing you to see more by moving your head... it's still just regular stereoscopic 3D. you need true holograms to capture light from all angles. Apple magic dust is really clouding your judgment, huh?

Thanks, I learned so much from this episode! Important question for you: I've been shooting on the Canon VR Lens and preparing 180º 3D content for viewing on the Apple Vision Pro. To play these video files in a 180º FOV on the Meta Quest, I simply press the button on the bottom right to change viewing mode, but how will I do this on the vision pro? Is it possible? I noticed that you played some of your videos shot on the Canon VR Lens during this video, but they were viewed within a regular window as spatial video. I'd like to view my content in its full FOV so that I can look all around.

Great review, and good information. I am not trying to knit pick, but PSVR2 also has eye tracked foveated rendering. Of course it is powered by a full PS5 and is not mobile, but it is also only a $550 device that requires a $500 console. The real problem there though is that PSVR2 doesn't even have software to play 3d videos yet... which makes no sense. The eye tracked Foveated rendering is huge though for VR gaming.

Amazing video Hugh, we are slowly getting there! I am not sure btw if VR180 can also be spatial on the Vision Pro btw. And what would be a small camera you recommend to capture spatial video (in a frame or 180degree)? I feel the Canon is too big and will scare the people I plan on interviewing

So to be clear, "spatial video" is some additional metadata that apple recognized, and adds some blurring around the 3d video/photo?

The Quest Pro also has dynamic foveated rendering (using eye tracking) and several games take advantage of that, most notably Red Matter 2. I believe it does it mostly on the software side, but still. Also, like others have said, that's not really parallax that you're seeing, IMO. You'll notice that the edges of the Spatial Videos are not only blurred but also closer in than the edges of the straight up 3D videos. It's like you've got a blurry picture frame that's blocking the outer edges of the 3D image/video. When you move your head, you can see some of what's being blocked by the frame, which enhances the 3D effect and gives the impression of parallax. That said, it's still a cool effect and very smart of Apple.

Another amazing video!

does Meta spatial video solution works as good as the experience on avp?

Could actual Spatial Video be implemented on the Meta Quest 3, do you think? The lack of parallax is quite noticeable in 3D video.

Could someone develop something for the Quest 3 that runs in the background so that it can be a simpler form of foveated rendering that locks to windows like when watching a video or using console remote play, thus increasing the resolution for what the user is most likely focused on at the moment then as the box goes away it spreads the resolution back out to the whole screen?? 🤔 in theory, from what he said, couldn't that be done on a Quest 3 since it doesn't have eye tracking capability?

You think Strip Clubs will let me in with one of those things on?

I'm not sure if there is parallax. It looks more like a crop and pan which gives the illusion of seeing more. I could be wrong though since I haven't tried it myself.

AVP 'spatial video' is standard stereoscopic VR180 (or other wide field of view) viewed through a narrower rectangular window. The ability to 'look around' is essentially shifting your gaze (moving the center of the VR180 image) to the side. It does provide some limited illusion of the ability to look around objects, but it is still essentially just stereoscopic video. You could produce a similar illusion by putting a standard 3D video behind a smaller window and shifting the projection to correspond with change in viewing angle. This form of spatial video takes immersive 3D VR180 and throws away both the immersive experience and a lot of the image. The advantage is that you can have multiple images open, it's a trade off between immersion and multi-tasking.

Gen 2 of AVP is going to be 🔥

Now we just need the spatial and panaromic bits to be merged so that we can move around freely around the moments we captured.

I think there's a reason why they used the word "spatial" and not 3D. As long as there's depth, then it is Spatial

So can we play our 180 Canon VR videos on the Vision Pro? I have one and I really want to get the full 180 to play on it.

Thank you for making this video, the other one in reference is SO ill informed it's ridiculous.

You showed your own videos you converted (RGBD?) , but bone that were natively created on the AVP. So the question you didn't answer is if the AVP and / or iPhone 15 can natively shoot in this spatial video format. They both have depth sensors after all. Combine the SBS image with additional low res depth data and some computer vision magic could in theory, yield pretty good results.

Right now all I want is to be able to play my insta360 x3 on my AVP.

Thanks for this video Hugh! So many Apple haters, even ones who don’t even have the device, trying to downplay the importance of this device.

Hugh, There is parallax in the content. You are just being fooled by the 3d window that is holding the content. I've been recording a lot of spatial videos, and have been really disappointed in the lack of any paralax. Open them up in the "immersive mode" and you will see what I mean.

PSVR2 and quest pro also have eye trackers

I just got back from my Vision pro demo. While the display is super sharp and the spatial video looked awesome I can tell you there is no parallax. Just visual trickery. The two cameras can not record the info behind the subject or outside the cameras fov. The only way to achieve that would be to deploy ai predictions or to remove the subjects from the image to record the environment in 3d. The tracking was good though. Can’t say it would be a decent purchase even if I had the money to blow though. I kinda got bored during the short demo and my face was already hurting from the weight.

VR 180 4k is much more immersive compared with 3d or 3d parallax? Why dont we make more VR 180 4k?

You rock, but this video is misleading. There’s no parallax with Spatial Video except for the frame. It’s 3D video in a cool wrapper. What app are you using to view 3D video on AVP?

The only real difference between this and normal 3D is just the player. It does some tricks to do that parallax and blurring in the edges. You should be able to say, build a video player that does the same thing for normal SBS videos, etc.

My Playstation 5 VR2 has eye tracking and has foviated rendering and HDR and headset rumble but it doesn't cost $3500

Wow, thanks for sharing this! You should also test this on Quest 3 after updating to v62. Try the same raw Canon file, plus also the converted Apple Spatial Video. That will be really interesting to see if Apple is doing some “special sauce” with their files.

I’m thrilled to see you’ve worked with ESPN+, I’m hoping you’re working with them on UFC because I’d love to watch MMA in VR180. I think the relatively small octagon would be easy to capture and the experience would sell the technology.

My guess is that Meta was going to release just a regular 3D movie player for spatial video, but then discovered this parallax effect on the APV and now are working hard on implementing it and that's why the update is taking longer to release. Fingers crossed

i dont think it really look like any parallax is happening it is just moving subtly

We need more spatial video playback example videos, the demo birthday video in the stores is decent but need more creative content

@hughhou Is Spatial Video captured from the stereo cameras of the Vision Pro better than the Spatial Video captured from the iPhone 15's single view lenses and then converted to stereo? I would imagine that recording in stereo would allow you to see around near field objects to reveal what is actually behind them. Side note: All those 3D 180 cameras you were looking at during CES are now obsolete. Unless they have depth sensors, they can't really make good Spatial Video and once people see it, they won't want basic 3D video any more.

Are they using some AI with the spatial video conversion then? Maybe that can be done with the Quest 3 too, unless there's a hardware limitation? Linking it to a PC would remove that limitation.

Bro... I sat here for months saying spatial video better have parallax, better have some level of 6doF if they are calling it that. Then everyone says nah its just 3d. NOW I hear otherwise. That's tight!

4:10 "this feature alone worth $3000", why? Playstation VR 2 also has foveated rendering and the cost is $550

XR4 - has @hugh Hou said it right? he showed the "Focal Edition" - but that is not meaning foveated rendering.. the Focal Edition does more. its essentially a passthrough with autofocus the foveated rendering also the "cheap" 3990 XR-4 can do.

As far as I can tell it's a stereoscopic video shown on a plane behind the canvas window frame. So I guess it is occluding part of your recording to add a foreground element that gives some 6dof interaction. The perspective in the video itself is baked in and won't change with perspective. So "spatial video" is S3D content shown in a see through window player.

Wow good information

i think certain ameteur science fiction sector artists started recording their avp spatial video for multimedia consumption mainly targeting male audience if you know what i mean :))

Hugh Hou, And this is why I come here! You do the research, you understand the tech, and are willing to help us understand. Thank you! So about that Spatial Video... Does it work by cropping what you see at first in the window, and as you move closer or around the window, you see other parts of the image? That's how I originally thought it might be working when Apple first showed the tech off. Is that right? If not, magic is a fun answer 😁 I'm supposed to get my Apple Vision Pro in mid to late February. I can't wait! And I also can't wait to see how you tinker and show us how to get the most out of it! Thank you Hugh! I really appreciate it! vgv

360 Vroomers is one youtuber who says it's just regular 3D movies.

LIKE YOUR CES 3D VIDEO HUGH!😊

I'm guessing Spacial video doesn't work with either 360 or 180 degree videos, just 3D side by side ones. Still, cool technology all the same.

Awesome

3D vid 3D vid, fuzzy frame

Perfectly stated!!!!

The Apple Vision Pro is so much better then the varjo it’s more comfortable weighs a lot less and is standalone also is way cheaper!

i honestly hope this video wasn't sponsored by... you know :)

Yes!

If the z-depth isn't capture by the camera then we don't have any spatial video. The video cameras need a Lidar scanner to create z-depth + video. I don't know if the new 360° cameras have this feature. Also the gaussian splatting technology is the future for 3d imaging.

Faux 6dof is still not 6dof.

I hate foveated rendering!! I need edge to edge sharp for my life :) Quest 2, doing this, some games doing this. A hate that!

VR 180 SBS 3D video is real life 3D just as i see in real world. No difference for me.

김치 귀여워

Sorry but I wish Spatial video was better than 3D stereoscopic video but it’s not. But you can keep selling it that way, just like Apple.

Sounds like BS to me. You can't capture data from different angles unless you have a camera at different angles. If the headset is in fixed position it can't do that. Only google light fields can do that and they record from multiple positions. Plus I struggled to understand anything he said.

👍

Who are you and what have you done to our Hugh? He would never leap from one gimmick to another without giving the cold truths. While you are calling out a "gimmicky 3D look" you are describing a "gimmicky spatial look". If there is now a process by which that stereoscopy can be refined and displayed more beautifully than before - great - but I think there should be clarity on the differences of captured versus computed depth data with no content creator hyperbole on either side. And our Hugh knows exactly why there is "the dreamlike look" - it's to hide the unfixable broken facts of life with 3D. This is not all gimmick though. There "can" be awesome here in some cases. There are 3 different levels of captured data that could be at play in a "Spatial Video" - 2 of them need computational tricks to be spatial (aka a 3D object in space) beyond "just 3D". A single camera lens and a depth sensor (lidar) is 2D+Depth and for many people that's already called "fake 3D" not full stereoscopy. The lidar gives you a measured indication of a point in space but the single camera means one eyeball's view must be faked. 2D+depth picture viewers like on Facebook let you look around an image in 3D because the depth data makes the captured scene essentially 3D object that can be pivoted. To use the modern jargon - Facebook has a spatial image viewer. Two camera lens and no depth sensors give true visual stereoscopy for the backs of our eyeballs but the location of points in space are left to computational guesswork. In traditional stereoscopy that computation is in the human brain. If you want a point cloud or 3D object to pivot around, a machine has to guess what points from each eye are at what depth. I have yet to see the Hugh spatial workflow but if a depth map is involved to turn the scene into a 3D object that you have some freedom of movement around - that's the point where it becomes a fake. Only with 2+ cameras and 1+ depth sensor do you get both the measured points in space for a 3D object and direct to eyeball image stereoscopy. The beauty of the Apple Vision Pro (and Quest 3) as a capture devices are the multiple cameras (at what IPD?) AND depth scanner. If and when those 3 streams of data are used to capture a scene as a 3D object with some degrees of viewer freedom - that's (almost) new and pretty cool.

How you pay 5k when the top one is 4200…

Every 3D movie, or any 3D content HAS PARALLAX. Thats how 3D works. To say that other 3D doesnt have parallax is nonsense that just shows you dont know what you are talking about.

So disappointing this spacial video. It doesn’t feel like Apple, it feels like a scam. I almost bought a pro 15 for that, I would have been furious when realising that by myself.

No, sorry, it's a simple relief video, in a frame that's in relief vision, so double relief, even if the second one isn't good, because you'd have to look at it from the front to see it properly. But it's a joke, especially with a little depth effect in the frame, to shift the relief video towards the background, with blur on the sides, because of course you have to fill the gap. The only parallax you'll see is when you close one eye and then the other, but it's better to do it from the front. Nice tweak, but it's all about communication and make-up, ingenious but totally feasible on a Quest 3. My first YT comment, after 25 years of 3D, but you have to set the record straight, it's pretty, it's a nice effect, but nothing revolutionary. Nothing to do with foveated rendering. Keep up the good work!

When you are communicating, take the damn thing off your face. My best and only real 3D camcorders are the JVC ones. Real parallax adjustment manually or automatically. You can watch the parallax change when you point the camera at different subjects. Nothing on the market that is affordable condo any of that. and it has a 3D viewfinder that shows 3D, left side, right side left and right sides, plus it has zoom lenses and stabilization. The camera lenses were spaced properly, unlike all the options available today. Since the Apple Video Pro came out the prices on these cameras have doubled and tripled. So a ten year old camera is better than anything on the market today and it was CHEAPER. Your take on spatial video is an illusion from software - a gimmick. The 3D from close lenses is gold for close up work.

So Apple still lied, because even though it's not just 3D, it's still nothing like their ads and promotional material (along the lines of the lenticular front panel advertising).

Meh I wouldn’t call it 3d more I call it crop and zoom with auto scrolling like they do with iPhones wallpaper soo yah it’s nothing new and yes it is a lie even if it small because a lie is well, just a lie..

Can’t understand this MFKA

So spatial video is worse quality parrallax video... I'll pass

Absolute clickbait title. It absolutely is 3-D, a two-image stereoscopic 3-D photo. Gimme an effin break. Apple knows exactly what they’re doing, and so do you.

It is absolutely nothing like your actual memories. You have absolutely no clue what you're talking about.

Click bate 😂

I like your content but brooooo, learn to speak englishhhhh, and I'm not a native

When people start believing it's rude that you accidentally bump into their 3D mesh, then clearly we're on the wrong track to saving this planet. Quietly wondering how many species disappeared from this planet while shooting the video. Just saying.

Saying Apple didn’t lie is a lie itself.