- Sony Interactive Entertainment recently published a patent to transform in-game visuals, shifting from the player’s character perspective to non-player characters (NPCs) perspectives within the video game environment.
- The method involves training one or more neural networks using images captured during human play sessions.
- These neural networks are then employed to create realistic videos from the viewpoints of non-player characters (NPCs) in the video game.
- To address computational challenges, the patent introduces a strategy where a single viewpoint is initially rendered.
- Machine learning algorithms are then used to efficiently transform this viewpoint into videos representing the perspectives of multiple non-player characters (NPCs) in a single pass.
- The patent allows for the customisation of non-player character (NPC) behaviours by selectively removing information from the transformed non-player character (NPC) viewpoints.
- For instance, reducing information related to hearing can simulate non-player characters (NPCs) with impaired hearing abilities, adding unpredictability and depth to non-player character (NPC) behaviours for enhanced gameplay variety.
Earlier today, we encountered a recently published patent from Sony Interactive Entertainment entitled “USE OF MACHINE LEARNING TO TRANSFORM SCREEN RENDERS FROM THE PLAYER VIEWPOINT,” filed in July 2022. Published earlier this month, the patent outlines an innovative method leveraging machine learning to alter the in-game visuals’ perspective, marking a big step forward in providing an incredibly immersive and realistic experience for players worldwide.
“Machine learning is used to transform screen renders from the viewpoint of the player’s character to be from the viewpoint of other non-player characters (NPC) in the room. One or more neural networks are trained using game images captured during a human play session, and then subsequently the neural networks are used to create realistic video from the NPC viewpoints,” reads the patent’s abstract.
“To avoid rendering multiple viewpoints simultaneously, a single viewpoint is rendered, and neural networks are used to transform it for the viewpoints of other NPCs in the area. A group of NPCs may be treated as a batch and a single viewpoint transformed to multiple viewpoints in a single inference pass. For cloud gaming the game video can be rendered/sent immediately for the player but the actions of the NPCs are delayed for a frame while the neural network generates the behavior.”
Traditional video game design often focuses on rendering scenes from the player’s perspective, making it challenging for artificially controlled non-player characters (NPCs) to make decisions based on their own viewpoints.
Sony Interactive Entertainment’s patent tackles a common challenge in the video game industry how to provide a realistic gaming experience for non-player characters (NPCs) by transforming the viewpoint originally designed for the player’s character.
The patent outlines a sophisticated solution: leveraging machine learning to dynamically alter screen renders from the player’s viewpoint to those of non-player characters (NPCs). This process involves training one or more neural networks using video game images captured during human play sessions, allowing the machine to learn and recreate realistic videos from various non-player character (NPC) perspectives.
To address the computational challenges associated with rendering multiple viewpoints simultaneously, the patent introduces a clever strategy. Rather than rendering multiple perspectives at once, a single viewpoint is initially generated.
Machine learning algorithms are then employed to transform this single viewpoint into videos representing the perspectives of multiple non-player characters (NPCs) in a single inference pass. This not only enhances computational efficiency but also provides a streamlined solution to the challenge of rendering multiple viewpoints simultaneously.
Recognising the growing popularity of cloud-based gaming platforms, the patent proposes a method wherein the player’s video is immediately rendered and sent from a cloud server to the display. Meanwhile, the actions of non-player characters (NPCs) are delayed for a frame, allowing the machine learning model to generate the transformed perspectives.
One notable aspect of the patent is its flexibility in customising non-player character (NPC) behaviours. By selectively removing information from the transformed non-player character (NPC) viewpoints, the patent allows for the simulation of non-player characters (NPCs) with different characteristics.
For example, reducing information related to hearing can simulate non-player characters (NPCs) with impaired hearing abilities. The randomisation of this reduction introduces an element of unpredictability, adding depth to non-player characters (NPCs) behaviours and enhancing overall gameplay variety.
Sony Interactive Entertainment’s patent isn’t confined to theoretical concepts. It envisions practical applications that can significantly impact gaming experiences. The transformed non-player character (NPC) perspectives can be input directly to a machine learning-controlled agent, influencing the behaviours of these characters.
Simultaneously, the transformed videos can be presented on displays, offering players the option to witness the video game world from the eyes of the non-player characters (NPCs). To ensure that the machine learning models understand and replicate human behaviour effectively, the patent suggests training them using in-game images captured during human play sessions.
Nevertheless, it’s crucial to emphasise that this currently exists solely as a patent, and its development or implementation is not assured. Only time will reveal whether and how the company intends to integrate this technology into its existing and future franchises.
What do you think about this? Do tell us your opinions in the comments below!
Similar Reads: Sony Improving Player Audio In Multiplayer Games In Real Time
Yes
No
Good job! Please give your positive feedback
How could we improve this post? Please Help us.
Submit
Started out penning short stories in my room, then stumbled upon my real passion for video game and technology journalism. Now, I’m all about playing and writing about them. With experience from VeryAliGaming, eXputer.com, and TheNerdMag, I’m currently dishing out news on Gamesual from my place.
