Sony's New AI Patent Predicts Button Presses Using Finger-Camera Tech

Sony has recently filed a new patent, WO2025010132, titled "TIMED INPUT/ACTION RELEASE," which aims to reduce latency in future hardware by leveraging an AI model and additional sensors. This comes in the wake of Sony introducing its first official upscaler, PlayStation Spectral Super Resolution (PSSR), with the PlayStation 5 Pro. While PSSR can upscale lower resolutions to 4K, the integration of newer graphics technologies like frame generation can introduce additional latency, potentially making games feel less responsive.

Both AMD and Nvidia have attempted to address this issue with their Radeon Anti-Lag and Nvidia Reflex technologies, respectively. Sony's approach, as outlined in the patent, involves a multifaceted solution. It includes a machine-learning AI model designed to predict the next user input, combined with external sensors such as a camera aimed at the controller to detect impending button presses. The patent specifically mentions, "In one particular example, the method may include providing camera input as an input to a machine learning (ML) model. The camera input may indicate the first user command."

Sony's patent filing emphasizes the importance of reducing the latency between a user's input and the system's execution of that command, which can lead to delayed actions and unintended consequences in gameplay. The company also explores the potential use of controller buttons as sensors, hinting at the possibility of incorporating analog buttons in next-generation controllers, a feature Sony has championed in the past.

While the exact implementation of this technology in future consoles like the PlayStation 6 remains uncertain, the patent clearly indicates Sony's commitment to enhancing gaming experiences by minimizing latency. This is particularly crucial for genres like twitch shooters, which demand both high framerates and low latency. As rendering technologies such as FSR 3 and DLSS 3 continue to gain popularity, Sony's efforts to mitigate their associated frame latency could significantly impact the gaming industry.