Sony Interactive Entertainment published new research on PlaySR on November 3rd, outlining an AI-powered upscaling system that combines motion vectors from game engines with optical flow to reconstruct higher-resolution images in real time.
PlaySR Combines Motion Vectors and Optical Flow
Games can render an image at a lower resolution and upscale it for display. The difficult part is keeping the reconstructed detail consistent when objects move or the camera changes position.
PlaySR works inside the game’s rendering pipeline and uses several game buffers. They provide colour data, information about how far surfaces are from the viewer, and motion vectors. Those vectors describe how parts of the image have moved between frames. Camera information also helps distinguish movement caused by the viewpoint from objects moving independently.
PlaySR uses motion vectors to align the previous frame rather than feeding them directly into its AI model. It also estimates movement through optical flow, which looks at changes in the images themselves. A small network called MaskNet blends the results based on which motion estimate is more reliable in each area of the image.
In cloud gaming, that rendering happens on a remote server before the video reaches your screen. Sony already uses a separate AI upscaler, QSSR, in PlayStation Cloud Gaming.
Sony Targets Ghosting and Blurring in Games
Shadows, reflections and transparent surfaces are especially difficult because the game engine’s motion vectors can be less reliable in those areas. When details don’t line up between frames, the picture can look blurry or show ghosting, leaving faint traces of earlier images on screen.
The researchers tackled this during training by adding synthetic on-screen interface elements to game footage and shifting them between frames. That gave the optical-flow model more examples of movement the engine’s motion vectors couldn’t track. They also trained optical flow separately before teaching the system how to combine both methods.
A compact network then restores finer details using an attention mechanism that works without additional model parameters.
PlaySR Performance and Future Research
The researchers tested PlaySR using two gaming datasets: QRISP, captured in Unity, and RDG, rendered with Blender. In QRISP tests that upscaled 540p footage to 1080p, PlaySR scored better than the supplied DLSS 2.2 and FSR 1.2 results across five image-quality and stability measures.
In a blind comparison using QRISP scenes, 75% preferred PlaySR, compared with 15% for DLSS 2.2 and 10% for FSR 1.2. In a separate test using six RDG sequences, 90% of 20 participants preferred PlaySR over the game-focused NSRR method.
Using optimized Python code, the researchers tested 4× upscaling from 270p to 1080p. PlaySR achieved about 34 FPS on an RTX 3090 and 40 FPS on an RTX 4090. Both results fall below the 60 to 120 FPS targets cited for competitive gaming.
The team also identifies frame interpolation and multi-frame generation as possible areas for further research using the same approach.
As always, remember to follow us on our social media platforms (e.g., Threads, X (Twitter), Bluesky, YouTube, and Facebook) to stay up-to-date with the latest news. This website contains affiliate links. We may receive a commission when you click on these links and make a purchase, at no extra cost to you. We are an independent site, and the opinions expressed here are our own.
