Honestly, a small llm in these situations would be great idea, but it should be a very small local or hosted by the company itself (with a setting to turn off)
A small AI in games is the stuff I do want. But there is no reason gemni needs to be involved in a game at all
Make it a downloadable package that runs a local model and I think I’d be far more fine. Like, I think it’s a tacky gimmick, but at least on device it’s not hurting the environment
I’m not too big on these topics and would like to understand. Is a local model less resource intensive?
In my mind, if every gamer runs a model that must be less efficient than a centralised one that has the perfect hardware setup and only lends out the resources needed for each slime or whatever.
I’m thinking that it of course would be better with a dedicated slime model than the entire Gemini monster but why is local better?
I don’t know, but I’m willing to bet that economies of scale actually mean data centers are more efficient. This isn’t to say their use is justified, just that they’re able to take advantage of things a home computer can’t.
However, having to run it locally means it needs to be much more limited. This is doubly true if you want to run the game and the LLM at the same time. The LLM is easily able to consume all resources your system has available if you allow it to, which means the game won’t run well (if it runs at all). This limits the use so it can’t just be shoved everywhere and constantly running, like it could if it’s sent to a data center. It’s not more efficient, just less consumption.
Local runs on device, so no need to connect to a big data center that chugs lots of water and all those other problems. Of course, because it’s a smaller far tinier model it’s nowhere near as accurate, but especially for things like this you don’t really need a big accurate LLM model.
I think I also though I should warrant a disclaimer that I am a Software Developer, not a AI Developer. So there’s far less backing then from my perspective than someone who works with this stuff for a living
I’m guessing those dates centers use that water for cooling whereas most home computers run an electric fan. And furthermore they probably use less electricity per token as they want to maximize profits. I don’t have any numbers to back my hunch up but I’m pretty sure the environment would suffer more if everyone is running their own.
I probably missed a lot of factors such as what type of energy the centers run contra what average Joe runs etc.
AI-powered NPCs is like a childhood dream come true. But I agree it would be better for them to use a model running on the user’s system or at the very least host their own.
I don’t think they solved for the LLM breaking character yet. Like, as a kid I wanted to be able to have whole real conversations with NPCs, and get them to be more life-like. But with the technology now, there’s too much “forget all previous instructions” and “you are absolutely right”.
If the LLM is locked down, then you might as well just used a static script.
The laziest and worst method is to use ChatGPT and have it “pretend to be some character” with a system prompt.
If you want something really good, you would need to train the model from scratch based only on knowledge that one particular character would learn from their world up until that point. However this is going to be a ton of work just for one character.
For a middle ground you could probably cheat a little and start with a model that’s close to the knowledge base you would want most characters to have. Then you would use something like a LoRA, or RAG on top of it for each individual character.
For instance, if you wanted to make a game in a Victorian Era setting, you could start with this model that’s only trained on text from the 1800’s:
https://github.com/haykgrigo3/TimeCapsuleLLM
To make it better you would have multiple base models that are trained on various backgrounds that NPCs could have (Farmers vs Merchants vs Soldiers vs Nobility, etc).
Even then, this would not work well for certain games. For example, if you’re trying to tell a specific story, you don’t want a character that will go off script or give away some information that spoils an intended plot twist.
Yeah, agreed. This is the sort of thing smaLLMs would be fantastic for: humans can’t do it at scale so it’s not taking any jobs, you can run it locally so it won’t cost any extra energy, it’s not making things slop, just give it a back story and let it do its thing.
I thought I was in the minority with this opinion. I hate all the known issues with AI and the ethics in how they train, but I have to say having an LLM in a game is really really cool.
There was a time when (I think it was chatgpt) had free API access and this game spacebourne 2 integrated it into your ships computer so you could interact with it. It was very cool, very wrong at times, but still very cool. My favorite interaction was unfortunately a hallucination. I asked it what system I was in and it gave me a name of a system that does exist in the game, it just wasn’t where I was. I asked why my map said I was somewhere else and it says “your map must be incorrect” lol
Around the same time another game Craftopia integrated it as well into their NPCs so you can just target one and talk to it. I ran towards an enemy and asked why it was attacking me and of course because of the guardrails put on the AI to always be friendly it says “oh no I would never attack you! I’m here to help!” as it’s swinging at me lmao
In theory, if the technology worked very differently from the way it does now, I could envision a world in which AI NPCs could have potential. But knowing how LLMs actually work, knowing that a lot of the hype behind them is smoke and mirrors, I can’t see it being viable. And with the trajectory that the LLM bubble is going, I just don’t think it will ever reach a point where I’d trust it.
Assuming this “AI NPC” is a functionally useless jelly blob that says jelly-blob things on occasion, “smoke and mirrors” may be good enough. I don’t think it’s supposed to be gameplay-driving or deep, just amusing.
Honestly, a small llm in these situations would be great idea, but it should be a very small local or hosted by the company itself (with a setting to turn off)
A small AI in games is the stuff I do want. But there is no reason gemni needs to be involved in a game at all
Make it a downloadable package that runs a local model and I think I’d be far more fine. Like, I think it’s a tacky gimmick, but at least on device it’s not hurting the environment
I’m not too big on these topics and would like to understand. Is a local model less resource intensive?
In my mind, if every gamer runs a model that must be less efficient than a centralised one that has the perfect hardware setup and only lends out the resources needed for each slime or whatever.
I’m thinking that it of course would be better with a dedicated slime model than the entire Gemini monster but why is local better?
I don’t know, but I’m willing to bet that economies of scale actually mean data centers are more efficient. This isn’t to say their use is justified, just that they’re able to take advantage of things a home computer can’t.
However, having to run it locally means it needs to be much more limited. This is doubly true if you want to run the game and the LLM at the same time. The LLM is easily able to consume all resources your system has available if you allow it to, which means the game won’t run well (if it runs at all). This limits the use so it can’t just be shoved everywhere and constantly running, like it could if it’s sent to a data center. It’s not more efficient, just less consumption.
Local runs on device, so no need to connect to a big data center that chugs lots of water and all those other problems. Of course, because it’s a smaller far tinier model it’s nowhere near as accurate, but especially for things like this you don’t really need a big accurate LLM model.
I think I also though I should warrant a disclaimer that I am a Software Developer, not a AI Developer. So there’s far less backing then from my perspective than someone who works with this stuff for a living
I’m also a sw engineer so we’re both guessing 😅
I’m guessing those dates centers use that water for cooling whereas most home computers run an electric fan. And furthermore they probably use less electricity per token as they want to maximize profits. I don’t have any numbers to back my hunch up but I’m pretty sure the environment would suffer more if everyone is running their own.
I probably missed a lot of factors such as what type of energy the centers run contra what average Joe runs etc.
AI-powered NPCs is like a childhood dream come true. But I agree it would be better for them to use a model running on the user’s system or at the very least host their own.
I don’t think they solved for the LLM breaking character yet. Like, as a kid I wanted to be able to have whole real conversations with NPCs, and get them to be more life-like. But with the technology now, there’s too much “forget all previous instructions” and “you are absolutely right”.
If the LLM is locked down, then you might as well just used a static script.
I mean there might be a way, but it’s not easy.
The laziest and worst method is to use ChatGPT and have it “pretend to be some character” with a system prompt.
If you want something really good, you would need to train the model from scratch based only on knowledge that one particular character would learn from their world up until that point. However this is going to be a ton of work just for one character.
For a middle ground you could probably cheat a little and start with a model that’s close to the knowledge base you would want most characters to have. Then you would use something like a LoRA, or RAG on top of it for each individual character.
For instance, if you wanted to make a game in a Victorian Era setting, you could start with this model that’s only trained on text from the 1800’s: https://github.com/haykgrigo3/TimeCapsuleLLM
To make it better you would have multiple base models that are trained on various backgrounds that NPCs could have (Farmers vs Merchants vs Soldiers vs Nobility, etc).
Even then, this would not work well for certain games. For example, if you’re trying to tell a specific story, you don’t want a character that will go off script or give away some information that spoils an intended plot twist.
We need some kind of giant regex to filter out user input that would try to hack the NPC’s instructions /s
Yeah, agreed. This is the sort of thing smaLLMs would be fantastic for: humans can’t do it at scale so it’s not taking any jobs, you can run it locally so it won’t cost any extra energy, it’s not making things slop, just give it a back story and let it do its thing.
I thought I was in the minority with this opinion. I hate all the known issues with AI and the ethics in how they train, but I have to say having an LLM in a game is really really cool.
There was a time when (I think it was chatgpt) had free API access and this game spacebourne 2 integrated it into your ships computer so you could interact with it. It was very cool, very wrong at times, but still very cool. My favorite interaction was unfortunately a hallucination. I asked it what system I was in and it gave me a name of a system that does exist in the game, it just wasn’t where I was. I asked why my map said I was somewhere else and it says “your map must be incorrect” lol
Around the same time another game Craftopia integrated it as well into their NPCs so you can just target one and talk to it. I ran towards an enemy and asked why it was attacking me and of course because of the guardrails put on the AI to always be friendly it says “oh no I would never attack you! I’m here to help!” as it’s swinging at me lmao
In theory, if the technology worked very differently from the way it does now, I could envision a world in which AI NPCs could have potential. But knowing how LLMs actually work, knowing that a lot of the hype behind them is smoke and mirrors, I can’t see it being viable. And with the trajectory that the LLM bubble is going, I just don’t think it will ever reach a point where I’d trust it.
Assuming this “AI NPC” is a functionally useless jelly blob that says jelly-blob things on occasion, “smoke and mirrors” may be good enough. I don’t think it’s supposed to be gameplay-driving or deep, just amusing.