this is great! Cant wait to see whats 2 papers ahead

It seems foot sliding is subjectivity very important because it's noticable to me even at this greatly reduced rate.

I didn't realize til way late in the video, it may not have been obvious enough - the ball is simulated with physics too, it's not just about getting the person to look like they're hitting it. They're actually hitting it. That's crazy

Electronic Arts is salivating looking at this

lol. Foot/leg sliding will need to be revisited as a concept when introducing Nadal's career on clay.

The animation still looks unconvincing, because the characters do not compensate for inertia during stops and direction changes. But that touch of realism would introduce the latency and might be not a good idea for dynamic games.

I have always dreamed of simulating soccer players of the past with video footage and it is happening. Exciting and frightening at the same time!

This has some significant ramifications for VR gaming, if anyone ever bothers to do the legwork and put AI and VR together....

This is TRULY amazing. I have experimented with MOCAP for years, and even with that, it's still incredibly difficult to make the motion look realistic when controlled in simulations. And that's because any person can quickly identify when something looks off. That's a natural ability people have because we see human motions every day, so everybody is an innate "expert" on it. And to put things in context, even expensive mocap suits still produce some level (or a lot) of feet sliding, not to mention the spikes that need manual cleaning. That alone in this paper is impressive enough to me! 😁

This is just amazing, when I heard it was only 4.5 hours of training my jaw dropped. It really did open my eyes to this being a possibility for the future of gaming. The only concern I have is system performance. Having this run in a game with all the graphics, assets, and modern day lighting makes me wonder about the reality of the potential.

4 hr is crazy... !

we should start filming top surgeons in 8k (+ myoelectric sensors)

Now they just need to imbue these simulations with caloric burn, muscle fatigue and the effects of repeat injuries on simulated characters. All things which would affect play style and play energy throughout a simulation/game.

Wow! Just imagine what would be just two More papers down the line!! What a time to be Alive!!!!❤️❤️

I’d love to see models trained off some of the great athletes of history in various sports and compare them to simulations of modern athletes in their prime play in 1:1 games, or dropped into modern teams, or assembled in dream team scenarios. All rendered out in crazy high end graphics and commentated on like a real sporting event

If they'd adjusted the grip it would've looked a ton better. Roger plays eastern forehand, not continental. And the simulation keeps the same grip between bh and fh

fantastic! another huge leap forwards in pose estimation which looks set to overtake mocap soon if this progress continues, and it is so much less hassle than donning a body suit or loads of motion sensors, which require endless re-calibration and are hyper-sensitive to magnetic fields. the foot sliding is indeed much reduced. in ordinary walking, running or (ballroom) dancing it might be more noticeable, but it is definitely getting there. as this paper is already nearly six months old, maybe there is an even better version already in the pipeline?!

Damn, being able to learn from raw untouched footage seems to be a very big deal to me. Think of how many hours of all kinds of motions we have.

adult films industry intesifies

Gotta love some Federer-ated learning!

As a tennis player and someone who's watched a lot of pro tennis, still obviously fake video but for AI to do this in such little time is amazing and this sets a good precedent for the potential quality of AI videos in the near future. Can't wait to see how good is next February of 2025

This will be super good for android motion. Also, I wonder if you can gain insight on who might win a game by simulating the players, than by following the brackets get more accurate predictions, making gambling less of a gamble?

so... the second paper down the line will be the ticket I'm buying to a robots-only soccer match?

still some sliding and fabric holding. but still super cool.

I love actually seeing a few papers down the line. What a time to be alive!

That's awesome. I hope it will have fast impacts in the sport video games world !

just imagine how many people ll be fired two more papers down the line

Great. Now that we can simulate specific players with high accuracy, we can make better sports bets by running a large number of virtual games beforehand to increase outcome predictability.

Would love to see if this works for all sorts of footage? Would Ninja Warrior work to get parkor gameplay?

This is great, but there is an uncanny valley aspect of this. If you've been watching tennis for a while, then Mr. Federer's stroke are instantly recognizable. However, there is still a floating sensation to these. Also, the gripping of the racquet pretty much is all done in one hand so it takes away from the effect but still way better than current video games and also the fact that AI created these motions, amazing

Oh man I really hope this makes it so live sports are accessible in 3D for anyone. The fact that no one has a VR setup where you can virtually sit courtside and replay things from any angle should be embarrassing to these multi-billion-dollar corporations, but they have monopolies so they don't care. This would really inject some competition...

holy cow! That's a lot of improvement from those earlier papers!

Am I understanding correctly that the ball is part of the model? I mean that the stance and motion of the character directly impacts the way the ball moves? That's cool, but I don't necessarily see how you'd use that for a video game; where would you get inputs precise enough for the player to control the character well enough to actually play?

there are some moments the head turns too much respectively from the torso.. but yes its getting closer to real movement.

5:53 that blew me away. There is real physics in this!? I thought the ball just was hardcoded to do certain motions but this thing is realistic just like in real life where just being a fraction of a second off in timing will result in a shank or hit into the bleachers.

This one is insane!! And it only needs 4h of video footage, nothing else, it sounds too good to be true lol. I really hope that this technology will soon be polished and ready to use for the public or at least that companies like Capcom or Squenix can implement it in their engines.

Wow so cool 😎 It would be amazing to create new niche sport games just like that

That is outstanding and it makes videogame development a way easier and less expensive too.

this is really amazing

sorry but we are not done yet to say that this simulation is so good that it looks like real world, but good job from the authors of the paper anyway

Morocco VS Spain Spotted !!! Li mghribi i dir J'aime

Would love to see something like optimus robot playing this well!

this is amazing! where can we get a implementation for this??!?

Interesting to see what would happen if the model is hooked up to a robot with the exact controlls...😮😮

Welp, it's not at all real yet. But it's getting closer, for sure! Can't wait for the next paper! Maybe 2 more papers until even I can't discern a simulation from reality anymore! But the progress is very promising and exciting! WHAT A TIME TO BE ALIVE! 🙌

Types of games I'd personally like to see this tech in: 1) MMA, since fighter styles and stances are sooo radically different, and games tend to just make a "one size fits all" type of body / stance / skillset; 2) Rally Racing games (yes, I'm imagining it can somehow extract the physics of the car just by tracking the movement): it's been 18 years since the last proper simulation was launched (Richard Burns Rally), and ever since you either get arcade games disguising as SIMs (I'm looking at you, Codemasters), or proper SIMs where rallies are not the main focus, meaning they're almost good, but not quite (Assetto Corsa, BeamNG)

Cant wait for Live AI Sports

Thanks for the video. The foot sliding... it is quite obvious.

The videogames industry that was already growing like crazy is now about to skyrocket

now have the tennissims not just on their backlines, but going to the net, jumping after balls and trashing their racks on the ground out of frustration when they miss ^^

Now we need it simulating Djokovic acting like Sharapova while playing against Maria Sharapova

with all these advancements in just 2 years, i cant imagine what new stuff will be out there in another 2 years

Pong is looking really good these days 🎾

now add some realistic conversion of the video game graphics to some TV like graphics and you have a video game you can't differenciate from reality. It should be doable with these kind of images, easier than with the cars we saw in the past.

Put this into a robot... and now we have a robot that can do everything by just watching a youtube video....

nice video!

So, who's going to build the first game engine that can generate these games for us based on our prompts?

It's interesting: I (still) can see it's not real footage, but I can't tell the exact part of it that makes me identify it as fake.

one day ill use this knowledge to create the most insane simulation to simulate everything and use it to develop movies, tv shows and videogames. Its like a virtual set that wouldn't have recurring cost of props and doesn't need an entire crew to film. And for videogames we just need to define behavior of the players characters etc making videogame development so much easier. Like a metaverse but actually useful.

Proper tennis video game needs a head set, a racket and an empty parking loot. Then I think it is possible for two players to reside in different parts of earth.

i can still see the foot sliding

Nobody moves off the baseline. Why's that? Is moving toward or away from the net more difficult to simulate?

It would be so great if the same could be applied to bird so we can have a simulation where we realisticaly play a bird.

now make tesla optimus play these games just watching humans playing

you'll need to play more games if that your barometer for looking like reality lol the foot sliding is _very_ noticeable and the interpolation between the movements are quite stiff. maybe two papers down the road we'll hit "looks like reality" ✌🙂

tbh this looks like a normal video game to me

Great, now when can I use one of the 500 papers to make an actual game. How can AI for game dev be moving so fast but also so slow.

NICE!

I just got my certification in animation and tech animation (including motion capture). Wish me luck.

Great, took way too long but still it's welcomed

Wow, we did survive this narration, phew😓. Congrats boys, we deserve a 🍺💪

They can use this to train robots. Imagine when the AI can watch all of youtube and learn to do all kinds of things.

even with minimal foot sliding, I can still notice something unnatural with how they move a bit too "lightly" but this is awesome compared to other methods

sports betting is gonna get interesting

The thing with foot sliding is funny to me, because there are plenty of scenarios where it is appropriate. For instance, do this same simulation training with footage of players on a clay court, and you'll see plenty of foot sliding, as you should.

I think you switch the before and after images in the thumbnail.

As a tennis player and watcher you can still tell its not real footage, of course, as your brain picks up on the small things that dont seem quite right even though it generally matches up with the footage. Still very impressive though! Edit: "only four and a half hours of video footage" - so one full grand slam match 😯

What a time to be alive!!!

Por fin vamos a tener un nuevo Wining Eleven a la altura!

Perfect they will be able to fire even more people.

Does anyone know what environment they are simulating this in? Unreal Engine 5, Omniverse, something else?

Ninja just claimed lerping between keyframes is "AI close to reality".

RIP motion capture studios XD , they had a good run at least XD

There are a lot of comments on this video talking about the underlying physics simulation and why the end result still doesn't look quite believable. And other comments about how this could be used to make robotics controllers. There is one huge caveat here. Since the objective here is graphics and not robotics, they use a technique called "Residual Force Control" as mentioned in the full paper. As far as I can tell, this allows the player policy to add any needed external forces (mainly to the feet) of the players in order to help the player do better and not fall over. In effect its cheating the cold hard laws of physics a little. There doesn't seem to be a limit on the amount of residual forces except that they are mainly applied to the feet of the player. It could be why the motion looks good but not perfect. It's as if the AI tennis players had the advantage of little tiny omnidirectional jet packs on each ankle. There authors also trained models NOT using these residual forces (there is video on the papers website,) and the motion is more believable but the tennis skills are a little worse because it's harder. Maybe if they used more video and more training they could get the policies without residual forces to also perform well and it would be closer to what we all hoped it was in the first place. Maybe in the next paper!

Not too far down the road, we'll just type in a plot, and a movie will stream out of the computer (play it as it's generating it).

With video ai can't understand bones movements and feet adherance. It's way better than before but we can do way better for the future !

Super!

If GPTs new video AI really uses unreal engine 5, you’d literally be able to generate games while your playing them, like let’s say your in a room and you say something to the npc, as you are saying it it’s generating the next scenes for the game based on the context of the game and what you say. Like imagine like gta 7 you can get put in prison, where all inmates are ai, you could join a gang, or find someone that’s willing to help you try to escape, literally anything you wanted to happen could happen in the game, like you could be in one city then decide to travel anywhere, even space, possibilities are as endless as the imagination

Should be a mere year or two until an AI is able to watch an entire football game while creating a live simulation of the same game. This could lead to viewers being able to choose any angle to watch a game at any time, just like a video game.

At this point, AI can really be applied in everything. Some years ago, not even a simple image could be generated correctly. Now, it can even simulate people playing tennis. What??

If only EA used this technology in their “sports” games.

I still kinda miss the videos where instead of the surface analysis you'd get a little more in-depth with the algorhytms.

It is impressive that the AI can infer the movements from video, but I have to disagree that it looks realistic. It looks no better than the motion capture.

All these new tools are going to generate so many amazing games that nobody will be able to buy because nobody has a job

this should be exactly the first thing that robots have to do with each other

I want this for boxing.

very good, but still feels a little weird, I think the speed of the racket might be mismatched with the speed of the ball, while the rest of the motion looks convincing, bringing it into the uncanny valley for me.

Will this be a playable videogame?

This is CRITICAL to the future success of robots. We can't be bothered with PROGRAMMING CODE to get them to do what they do, like we have with robotic arms in factories in the past. They need to LEARN BY WATCHING, and of course already are as 2 students demonstrated in the real world with off the shelf components and a laptop... IMAGE interpretation is critical to success. It is how we will teach AND how they will recognize success in their tasks, or the need to alter their task (such as when the researcher moves the object they were reaching out to move). When we perfect SEAMLESS and effortless near perfect quality of movement of information between real and virtual worlds for use in robotics for learning we will really be on the way. I predict around 2026 you'll start seeing the public hit with the advertisements saying "robots are SAFE!" as they start gearing up for mass production and need to quell the iRobot fears. By 2030, you'll think nothing of driving past lots of them out working or commuting to complete assigned tasks.

Undeniably a significant technical stride, but between the foot-sliding and pre-2012 fabric simulation this doesn't even approach the 'ballpark' of actual verisimilitude. Your hyperbole at 2:45 is fatuous.