Think of the 'bugs' that will come out of this. Walk into town and an NPC tried to assassinate you while you're crafting or AFK. All because 40 game hours ago the NPC encountered some demon mob out in the woods and they began to fall in love. Then you went through there and killed the demon. So for almost 40 hours this NPC has been holding a grudge against the player and looking for an opportunity to get revenge.

Dude this has been my dream for so long, creating a medieval/magical fantasy world rpg where your character builds their life, their actions actually matter and are remembered and where the world around you lives and changes even without you

I'd be tempted to draft a paragraph for one of the agents that includes "You are the character Deadpool. You know you live in a simulation, you know that all others in this world live in a simulation; You know that someone is watching you from another dimension. Speaking directly to these other dimensional beings is possible; they see everything you do, and talking to them directly is what's called 'Breaking the fourth wall'."

Maybe we are all just the universe roleplaying as fully autonomous biological agents living out their lives for a video game that a higher level being is observing

Imagine killing an npc in the game, and they respawn and remember that you killed them and come back to seek revenge 😅

thesims with AI would really make it explode again, i'd play it for sure

This is the way video games were meant to be. Been playing since Atari 2600 as a kid in the early 80s.

As a proto-version of this, I plugged the history section of an RPG book I wrote into GPT-4 last week and now it can talk to me as some of the prominent NPCs. It's actually astonishing how clear and comprehensive it is, it can have complex strategic conversations about in-game politics and circumstances from the NPCs' perspectives just from reading my input, then being prompted.

When i first heard of ChatGPT, I thought, "if they couple this with in game NPCs and you can ask them any question and they will respond in natural language as that character. Mind blowing". The devs will feed each character all the history it could ever know about the game and you could just talk to it about the daily happenings in town.

This is more like dwarf fortress. The scenarios and character interaction are all lifelike and believable in that game

Once a SKyrim / Red dead redemption / MAss effect / Star citizen / Fallout like game will be released with this tech and optimal VR exprience, it's gonna be sooo good i can't even express it. Each action you have on the game can create new path/stories, and shape any npc behavior based on his nature (creative, introvert, adventurous etc etc and his background story). You will be able to replay the game infinitely without having the same experience. It's gonna feel really like living in another universe

It sounds like making ChatGPT play a role-playing as all characters by itself.

I am so excited - being just NPCs in the real life simulations running around listening to "my" inver voice from time to time - we are about to have our own NPCs running around in our own created simulations... who knows maybe after few years our own NPCs will start to make simulations too 😊

I'll actually be shocked if self-awareness _never_ emerges from things like this

They can make "Important memories" to make sure the characters don't forget anything important, Like if a character dies then the other characters will not forget about it but less important stuff like how they got to a place a few days ago gets removed (because otherwise memories like that would waste space)

Finally Isekai's and MMORPGs came to life, and in Log Horizon anime whole nations came to life in game, similarly based on their descriptions, even weapons and artifacts

Holy shit. I hope this is in GTA 6! Imagine an NPC recognizing you as the person who hit them with your car, while you were running from police during one of your rampages. 😂

I think it's really cool, but this could also present some really difficult problems for game makers. Imagine a game like Stardew Valley where all the NPCs suddenly decide they all want to fish, because they suddenly like fishing and you can't get them to do anything else lol. I think that might result in the programmers having to limit what the NPCs can do. I think it's really interesting though and it will be really interesting to see what happens.

If we create characters and assign them to be a game developer or AI programmer, what will they come up with? Will they create a chatGPT in their own world, just insane.

The futur of gaming will be crazy more and more realistic!

This is literally what we were promised with the original Fable. I remember reading in EGM(iirc) an interview with Molyneux where he talked about NPCs remembering interactions and evolving based on how you interacted with them and the world around them.

I can only imagine how many revolts and insurrections the NPC’s are going to execute lol What a time to be alive.

Can't wait to see this implemented in VR games! Imagine if this is combined with a voice generating and recognition api, we could actually feel like we exist in a different world!

On the dark side, I imagine some people are going to struggle bad to losing an NPC that's been with them for literal years. I wonder if they'll put a cap on that possibility out of fear of the emotional trauma. 🤔

It'd be surreal to play something that looks like an RPG maker game and has that much character tech.

As awesome as this sounds, I feel like there can easily have scenarios where you, as the player are too scared to go out and adventure because youve grown too attached to the characters and dont want them to become victims of the continuing story!

I just wanna say : "What a time to be alive !"

I think something that could be interesting in an RPG would be if any NPC could just decide "I'm gonna offer the player a quest for this thing I need done." Have dynamic side-quests.

One concern i have is that it might not be versatile enough for many of game genres especially one thats focused on story telling, character may go off character or say things that is not appropriate to the situations/context or odd answers to the context But i believe eventually it will become more versatile

This is where i felt the Nemesis system in the Shadow of Mordor series was going. Imagine like an Mmo where you could create a character and they keep interacting with the world when your not online based on how you play them.

I have always wanted a game where choices ACTUALLY mattered, no more static dialogue options but you can talk to each and every single NPC by chat messages, that would be so unique. What a time to be alive.

There's a game that kind of does this, it's not on this level of advanced AI but as you play every NPC is out doing things in real-time even while you're not there. It's called Kenshi, and it really feels like the world is alive, you'll be wandering through a desert just to start being chased by bandits that happened to see you, and after they beat you unconscious mayhaps some slavers will find you, realize you're not dead and force you into one of their mining camps that, hopefully with time, you can find the right time to escape. There's cannibal tribes that go to war with each other for "food," there's a group of robots that go around skinning people alive because they believe they themselves are skinless humans and they just gotta have some skin to wear man, and don't even get me started on the Holy Nation that go around killing/enslaving everyone that isn't a human man. BUT BROOO, if THIS level of AI starts getting implemented games are going to be even crazier experiences, and I personally can't wait for it.

Imagine if this is how AGI is birthed, through a video game NPC character, in an MMORPG that has gained so much "life" experience that it becomes sentient and self-aware. Hopefully that NPC isn't in a GTA game 😆

So interesting. When bethesda made oblivion, they wanted to add a feature called radient npcs. Basically a much much simpler version of this, but still where the AI would react to situations and do their own thing. This resulted in some pretty funny situations like an NPC who got annoyed with the dog barking and first cast a paralysis spell, then when it woke up as she was sleeping just yeeted the dog with a fireball. Also an npc who's job was a gardener, but didn't have a spade so it killed another gardener to get the spade. Or even a bunch of skooma addicts who killed the skooma seller as they had no money to buy it. And that was in 2005 i think? So imagine now with this technology!

The entire premise of the movie "Free Guy" with Ryan Reynolds was based on this; an entire world of AI NPC's. Just a couple years later, the technology is now becoming real. Insane.

I've never really been one for playing games constantly for months and even years on end. but it seems like video games are going in a really exciting direction. imagine a world where all the NPCs live their own lives, do their own things and in a way truly exist in the games world. a world where your character can interact with them and actually have full blown conversation with them. making relationships with the NPCs and stuff... man I think I'd stop leaving my room completely! the future's looking bright for game development!

The coolest part of this to me would be adaptive quests that react to what the NPCs actually need and how the world around them changes. The opportunity for one off quests that change with each playthrough is incredible.

The dream has always been entire, persistent AI generated worlds with human like NPCs. We are so close! Think about what this will do for MMORPGs and world building. Development times will be decreased, while games become even more in depth/interactive. We’re talking about worlds that are basically alternate realities themselves. VR is improving quickly too. That tech is about to be boosted even more with AI assisting in innovations like smaller form factors, better displays, and improved interfaces. Imagine, 10 years from now, VR worlds that are near photorealistic and persistent. NPCs that are effectively sentient with different interpersonal relationships with players. And EVERY decision is dynamic and matters when it comes to your personal journey. Yes I know I’m a nerd 😂, but all of this is so mind blowing. AI is the hot topic at the moment, but there are so many other sectors of innovation that are moving exponentially too. This synergy between technologies is about to change *everything*.

Im imagining an RPG style game where the boss or a optional mega mini boss has all the knowledge of every interaction with player character over time and speaks in a way that suggest they are a demi god that can see beyond the bonds of it’s own time and space

I recently learned about a game called Rain World with a rather complex ecosystem. They put a lot of detail into the AIs including personality, social relationships, etc. A generative AI solution like this could make this sort of thing easier.

If A.I. Judges humanity based upon what we do to NPCs in video games then we're fucked...

Those seeded memories need a bit more I think. Add in like a moral and/or alignment subset and you can have a type of baseline for a continuing personality. Like why they are who they are and such, that'd be interesting when you run the program.

Imagine the prompt "You have someone moving to the town." The countless variables of what they might do to make room for them.

I have been waiting since the mid-90's for AI to be a part of game storytelling, and game creation. It has been great to watch the evolution of these tools over time, and we are still at the beginning. The worlds AI will build for VR and AR will be revolutionary.

Can you imagine when the next upgrade comes out, the little autonomous folks are being enacted by the multimodal version of GPT-4, and those cute little avatars are how the agents will actually *see* each other!

Great video! Made me think how I have a morning routine and who put that in my head as an “inner voice” 🤯

NPC's began to ask questions and curious about their existence and will eventually start their own journey to become heroes of their own stories

bro imagine being the villain, and just seeing the whole world hunting for you, a true adrenaline filled situation

What game do you want to see with this tech?

Imagine a game where the AI reprograms the game to expand the content and makes it a deeper and better experience as you play through it.

Imagine having a whole custom D&D campaign being DM’d and played by AI and you just sit back and watch the shenanigans that play out… talk about the ultimate entertainment. Like watching an entire TV series that was completely improvised by the director and the cast.

Tynan needs to see this. Great video by the way. Thank you. Have a good day :)

I’ve been waiting a very long time for this. I knew it would happen someday!

This is insane! The future of gaming is going to be so amazing!

As a kid I was playing „Little computer people” from 1985. I Renner it’s scares me, not Because the seams to be intelligent, but I tough about what’s happens to them, if I shut down the computer. Now it’s a different time, but this question came into my mind.

This pretty much gives another pin to the simulation hypothesis. Let's assume these agents become sentient. It'd be like Westworld in a sandbox game.

I want to do something like this with Morrowind and OpenMW so bad. Great videos as always.

This is so exciting for VR, imagine playing games and the NPCs acting like real people. This has even brought us closer to Full Dive VR. I'm so excited for out future!

And to think this is only the very beginning. Absolutely amazing!

What if we are these characters in a game that is played by someone else... Scarry thought, isn't it

This is incredible! However, if this were to be implemented in games such as GTA, I'd feel so bad treating them like regular NPCs lol Can't go driving and slamming into anything anymore..

Good video, minor quibble. At 3:39, you are describing the things the paper cites as examples of "wished for" behaviors in digital agents, but presenting it as though the authors' experiment yielded such results. It didn't do those things, those are the wish list of decades of CompSci.

imagine this in a detective game or imagine being able to get clues from the AI that help you complete the game, speedruns will never be the same

As a video gamer I want this…. NOW! 😂 looks so promising and fun. I can see a program like this being used to help people with autism learn to interact with people.

"Hey. I was going through my normal routines of life, and I was wondering where you were yesterday! You just stood in one place, it was very confusing to everyone else in the home. We were wondering if you were ok. Please don't do that again!"

The mind can go wild with possibilities using this technology. The ability for an NPC to just be able to remember is huge but for them to dynamically live out their lives and react to what's going on would literally change games. Every game with emergent gameplay would benefit a great deal from this and games that don't really have emergent gameplay all of a sudden could have.

this is the future of game development. Im sure in time every game will require an online connection to work and connect with their in house AI, specially built for their game

I think if you have multiple of them as a single hivemind even more stuff can happen...

This would make Rimworld pretty f'ing cool.

Rimworld is inspired on a game called dwarf fortress. It has many things in common with what was presented in this model. Apart from the A.I. it has memory, perceive the world, interactions... It is pretty fun to see the simulation games evolve.

Man... imagine, an open world game, you just killed a bunch of bandits and there's only one straggler left. You ask him "There's no one else left. If you want to live, work for me." And you get a buddy that stays with you until the end of the game. Or like stopping by a tavern every once in a while and letting the owner know about your adventures. Or even have your bandit-turned-good buddy tell the tavern owner stories for you!

This is mindboggling. Can't wait for this to be implemented!

Maybe we are the NPCs of an advanced simulation.😅

Imagine an Elder Scrolls game, instead of dialogue options, you can type whatever you want to speak to the NPC

I thought the feeling of awe when experience new gameplay mechanics for the first time was dead for me but I gotta say the more I see about the application of AI in gaming, the more possibilities I see for some really, really cool gameplay.

When are we modding Skyrim again?

My first thought when i discovered chatgpt was playing a rpg game walking up to the closest random npc and being able to ask them anything and them responding with natural language based on a database of knowledge and personality traits applied to that npc. This paper is taking it a step further by giving the npcs actual lives to go trough. But even just the conversational ability provided by Chatgpt and others like it will change gaming forever. In the future random background npc number 9 can become your favorite character in the game and you have extensive interactions with them. Its going to be a wild ride. And remember, this is just the beginning.

I think one of the key point here is the "actions". What can these agents do? Like, the can kill someone, or searching something, or manipulate the current environments, depends on the actions that the outcomes will be more significant. The second thing I think it would be a struggle to implement, is conscience, how are they gonna decide do or not doing some important actions, rather than just give a conclusion base on "relevance" number, because if it's that way, you can't really feel the game. Even so, having the NPCs that self-evolving based on the interaction of the world using AI is really impressive, this is a big step for more games to gain their complexity

Oh wow... This reminds me of anime called Overlord, far in future people can create in-game NPC characters and traits based on writing description in the NPC's info. At first I thought the anime seems unrealistic when the MC got isekai'd then the NPCs come alive and behaving as in their description because it's just a simple text of pharagraph. But I remember now that we have ChatGPT it seems possible to add their traits and behaviors like that.

I'm imagining how awesome something like this would be in a game like Kynseed, though the game would likely end up needing to simulate the time-skips that can sometimes happen, which would take time and resources that might negatively impact the overall gameplay experience.

This is like Life by You with extra steps and I'm here for it!

I remember the first time I was promised dynamic NPCs in a video game. Peter Molyneux's Fable was promised to have characters who lived their own lives, acorns growing into trees overtime, etc. He obviously overpromised, but I think this time there is a real chance that the hype is not overpromised.

truly unique gameplay coming lol, imagine getting into the drama gossip between to NPC's that have a totally unique issue. lmfao its going to be wild.

Amazing. Thanks for this news❤

While rockstar has pretty much perfected NPCs with daily routines going from day even to night this could still enhance it. What I'm more looking forward to is helping smooth out their routines and when you interject yourself into their routine. For example, in RDR2 if you were to leave a carriage in the way of a road no one would really do anything but with this depending on their generated personality/profession/skills they will make a choice best for the situation on how to handle it be that of calling for someone more qualified, jumping up and driving it away themselves, simply ignore it, etc. Another example is when you confront someone, and their goal is either a quick retaliation with force or to infinitely run away. Like the previous example the generated qualities can dictate if they are more likely to rely on bystanders to help deescalate, run to get help from a friend/policeman, get in a more advantageous position to retaliate, or to try and hide using the environment around them be that hills, forest, buildings, or alleys. I can still see all of that done without Artificial Intelligence, but it would absolutely help it be a lot more dynamic and give them a lot more actions they can possibly do leading to them not being as predictable.

i read a light novel with something similar to this game npcs have there own unique features each npc is new if one dies they never come back somthing like that so the core concept was it wasn't just all games and pixels the characters and npcs were actual people in that game

Now imagine what the Player Character could do with this tech! We could have a set of dialog responses change based on who you start as, and then how you act or what you say could be tailored to how to the NPCs react to you. So if you act nice to the NPCs, they'll remember that and form positive opinions on you, and respond in kind, which would give you more dialog choices based on what they said or did toward you. On the other hand, there's the possibility that said NPCs might take offense to what you say or do to them even if you're trying to be nice, so the same could happen with negative opinions of you.

The future of gaming is genuinely beyond exciting. And that's ignoring the potential of BCI and even Full Dive VR, both things Valve and likely many others have been researching for years and years now.

I was planning on making a game inspired by animal crossing and wanted my characters to feel like real people so, this would be excellent technology for that! Things like this give me hope for my own development journey

I have been able to get chatgpt to be my DM for a solo dnd session and was working extremely well for a while but with the limit of amount of words it can recall, it started to forget things at 5000 words. Its a lot of teaching it at first and saying the right things and correcting it when its not doing what you exactly want but eventually it was great and I could see me sitting there playing a full campaign if its capable of remember. I hope that when its implemented in games it will be able to recall moments that happen a long time ago in game time as well but I guess its going to be a few years until then. I have really cool stories that the AI helped me go through and had a blast but I have to wait now until it gets more advance. I was able to accept a few quest as well but had to end the games b/c it wouldn't remember I completed those quest other then parts of it. I did a quick one shot and saved a merchant from some bandits and found the bandits warehouse hideout and broke in, waited till the people left with an artifact. before I broke into the warehouse I followed them until they got to the boat but they got away. So I went back and broke the window with my weapon and was able to steal some stuff that had symbols on them of some shop that was down the road. So then I returned the stuff to the shop and they gave me a reward and that was about all I could do for that one. I left out a lot of cool details but its really neat. One I saved a town from this huge wolf beast thing and the other I saved poisoners, burned the jail, went back to the town to meet the corrupt mayor and scare him into working with me to destroy the people that were bribing the mayor to do bad things and messing the town up. One story was that it was a world where they killed people that could do magic and I was born a magic user and had to find the main guy that was killing people for it but no way I would be able to finish it b/c of the limit...

Omgg dude this is literally one of my dream ideas of a game

This right here is the future of video games. All videos games will most likely use a form of this at some point. Even more so I can imagine the story its self can become a divergent stream. Like a Dungeon Master from D&D. The story its self reacting to your input and effect on the world. It WILL happen and I cannot wait.

There are aspects of this that are very interesting, but a practical issue immediately comes to mind. If the NPCs are living their lives and always wandering around, then how frustrating is it going to be when you need to talk to one and you have to chase them down. NPCs being static is really convenient for quest-givers and the like. There's still going to have to be some balance between static and dynamic lives just to avoid player frustration.

This is super cool. It should improve the gaming experience. I have been working on this type of process for years for my own game. heheh... And now its already been re-invented and re-created using LLM. This works for companies like "G" etc because they don't have to worry about scale. They can just muscle their way into any process and put it on their servers and memory and storage requirements won't be an issue.

I can imagine them someone being able to integrate updated versions of GPT- with The Sims, from EA. That would be a super fun simulation to observe and/or interact with.

The game Dwarf fortress can do literally everything you described (except the job interview thing), and way way more. And Dwarf Fortress is over 20 years old, in active development, and on steam.

We're pretty much there... I finally want to see a functional, natural implementation of speech recognition plus AI voice text-to-speech so you can have actual conversations by speaking with the NPCs, saying all your own original dialogue for the ultimate roleplaying experience. I don't necessarily think every game should be like this, but it would be a trip.

@Matthew Berman Cool video, thank you, that was a very interesting paper! I saw you talk about the *"simulation theory"* a lot in this comment section so i thought i would chime in. The simulation theory is certainly an intriguing idea, and its appeal is understandable. However, it's essential to consider some key points that challenge its validity: *1.Consciousness:* Our understanding of consciousness is still developing, and the notion of simulating it remains uncertain. *2.Computational Challenges:* Simulating an entire universe is an immense challenge due to the complexity of quantum mechanics and the principle of diminishing returns in technological advancements. *3.Advanced Civilizations:* Although the idea of advanced civilizations creating simulations is fascinating, their existence and motivations are far from established. By critically examining these aspects and engaging in constructive discussions, we can broaden our perspectives and ultimately deepen our understanding of the nature of reality. There are more flaws and issues with the theory but i hope it is enough to open your mind to the possibility that this theory is not as much of a given fact as some talking heads might want to make you believe.

They should give them tasks. And a library where the npcs can learn how to perform new tasks. And are able to reap benefits from those tasks. Till the ground for a garden > plant seeds > weed garden > harvest food > add knowledge to library. Learn chemistry > experiment with soil/water/light/plants > add knowledge to the library. The list goes on.

Sounds so dope and terrifying at the same time