• Sony is internally testing an AI technology where the player could talk to PS5 characters like Aloy from Horizon Forbidden West and get in-game, real-time responses.
• The technology reportedly uses OpenAI technology to replicate thinking and reactions.
• For now it is being showcased to Sony execs as it is being developed.

There is a potential that future PS5 exclusives like in the Horizon series, Death Stranding, The Last of Us, or even God of War could see characters enhanced with AI so that they can speak with the player and have more natural-looking facial animations and emotions.

This is according to The Verge which says it was anonymously sent an internal video from PlayStation that showcases the use of AI technology in a version of Aloy, the protagonist from the Horizon series, and in particular Horizon Forbidden West.

The video, which The Verge published on YouTube but was later taken down by a Sony affiliate, is narrated by Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment. It showcases an AI-powered Aloy that can hold conversations with the player through voice prompts the player gives during gameplay.

In a real world scenario, this would be done through microphones on the PS5 controller or through a branded headset.

“Aloy could be seen responding to queries with an AI-powered synthesized voice and facial movements, both in a demo setting and within the full Horizon Forbidden West game,” the report reads.

Raghoebardajal explains that the showcase is of a prototype that is being tested internally by Horizon developers and Sony studio Guerrilla Games and is meant to demonstrate the technology to company executives and affiliated developers, like Naughty Dog and Sony Santa Monica.

Per the video, Aloy is powered by OpenAI’s Whisper product for speech-to-text, as well as both GPT-4 and Meta’s Llama 3 for conversations and generative responses. Sony is apparently using its own in-house voice synthesis system for realistic inflections in the speech generation.

“This is just a glimpse of what is possible,” says Raghoebardajal, with the demo running on PC, but expected for latest consoles like the PS5. The showcase is of technology that has been in development for a good while, and it is claimed to have been first demonstrated to Sony about a year ago.

Sony is likely sitting tight on the footage because it raises a few serious questions: firstly, what happens to voice acting veteran Ashly Burch when Aloy becomes powered and voiced by an AI? The same goes for any other character that Sony may be thinking of using this technology for.

What about the writers? When generative AI is coming up with the responses for a character, do you need dialogue writers? Or rather, do you need as many dialogue writers?

Aside from these questions, what would the gameplay even look like for this? How could the player speak to Aloy when they’re supposed to be playing as her. If Aloy is just a stand-in for the showcase, would the future see the player speaking through Aloy to other NPCs?

This unfortunately could lead to a lot of NPCs saying things like “Sorry, I’m not sure what you mean” when asked if they “be actin’ like a sussy baka.”

AI: another A in AAA gaming?

Sony and PlayStation are definitely not the only big names in gaming leaning towards AI. NVIDIA showcased a very similar AI-powered technology for games where players too were able to interact with NPCs.

Microsoft reportedly has been looking into this tech from NVIDIA, called Inworld AI, to bring it to more games in the future. Perhaps there is a potential we could see it in the next Elder Scrolls game? I shudder at the thought.

Ubisoft had to deal with vitriol from players after it revealed it was using an AI software in game development to quickly come up with in-game barks for NPCs. While others like Nintendo don’t even want to go near AI with a 10-foot pole out of fear of copyright infringement.

The technology of generative AI will likely be used more in gaming going forward, we just hope it’s in tasteful and useful ways to make the lives of developers easier and doesn’t see them getting fired.