Glossary:High-fidelity upscaling

From PCGamingWiki, the wiki about fixing PC games

For a list of games, see games with high-fidelity upscaling.

This article lists various high-fidelity image upscaling technologies used for video games.

Deep Learning Super Sampling (DLSS)[edit]

Deep Learning Super Sampling is a set of image upscaling algorithms developed by Nvidia for real-time use in video games using deep learning to upscale lower-resolution images to a higher resolution that is accelerated through specialized hardware only present in the RTX series of GPUs.[1]

DLSS was first announced with the release of the Nvidia RTX 20 series in September 2018 and the first titles featuring it releasing between then and March 2019, relying on per-game training and only featured in a handful of games with generally subpar results made DLSS (1.0) at the time rather disappointing and not recommended. An improved version of DLSS unofficially labeled as 1.9 was shipped with the game Control in August 2019 providing better image quality than 1.0, although still generally worse than a native image of the upscaled resolution.

DLSS 2.0 was released in March 2020 providing much better picture quality, performance and adjustable quality options over previous versions, and since it didn't need per-game training it was implemented into many more games and is now generally recommended if you have a supported GPU.[2]

DLSS 2.1 was released in September 2020 providing support for VR [Note 1], and introduced the Ultra performance quality option, meant for 8K gaming with the RTX 3090. Later versions of DLSS wouldn't provide much benefits as compared to before, mostly meant to fix issues with ghosting on fast moving objects, and these versions can be used on older DLSS 2.0 implementations via replacing a single .DLL file.

DLSS 3.0 was announced on September 20th in Nvidia's September GTC 2022 keynote keeping the same upscaling algorithm that DLSS 2.x.x utilizes while adding two more features that being AI Frame Generation and integrating Nvidia Reflex. Frame Generation is touted to massively increase framerates as shown in a demo of Microsoft Flight Simulator (2020) running at over 100 FPS with DLSS 3.0 as opposed to 60 without, although Frame Generation has drawbacks like it being only possible on the 40 series Ada Lovelace GPUs because of a new optical flow accelerator and it increasing input latency significantly. Nvidia Reflex is being integrated into DLSS to lower to the increased latency given by Frame Generation, much in the same way it reduced latency when used normally. Both Frame Generation and Nvidia Reflex alongside the original Upscaling algorithm will have toggles allowing users to enable any of the features as they want. DLSS 3 rolled out into games and programs in October 2022, with Marvel's Spider-Man Remastered being one of the first games implementing it on 12 October, 2022. [3]

FidelityFX Super Resolution (FSR)[edit]

FidelityFX Super Resolution is a set of image upscaling algorithms developed by AMD for real-time use in video games, using traditional hand-written algorithms to upscale lower-resolution images to a higher resolution. It does not use machine-learning algorithms, unlike DLSS, to upscale the image, and instead relies on spatial upscaling on traditional shading units (FSR 1.0), and thus doesn't need specialized hardware to function and can work on a large amount of GPUs.

FSR 1.0 was released in June 2021 with a handful of games supporting it, and it generally provided a decent image at higher resolutions and quality presets (like 4K Ultra Quality), but lower resolutions and quality presets usuallyu give a subpar image (at 1080p, or using the performance quality preset), so most would generally advise using the Quality or Ultra Quality presets at 1440p or above for decent image quality, although do note that FSR standardized Presets are not enforced so some titles such as Dota 2 offer resolution sliders instead.[4]

FSR 2.0 is a new version that changes the upscaler to a temporal one similar to DLSS, touted to have better picture quality than FSR 1.0 while still not using machine learning algorithms, being open source, and supported on a wide range of hardware. Announced on March 17 2022 alongside Radeon Super Resolution and Adrenalin 22.3.1, it released on May 12th 2022, with the first title supporting it being Deathloop[5] and the source code being published on June 22nd. [6] Reception for FSR 2.0 is positive as it has much improved picture quality than FSR 1.0 and can come close to DLSS 2.x's while still providing a similar performance uplift and working on non-RTX GPUs, but falls short of some fine detail or moving objects when compared to DLSS or Native image. [7] FSR 2.1 was released on September 8 2022 promising improvements to overall image quality and fixing ghosting and shimmering issues with Farming Simulator 22 being the first game to support it.[8]

FSR 2.2 and FSR 3 were revealed during AMD's RDNA 3 annoucement. While FSR 2.2 is a minor update said to improve ghosting issues and first to release in Forza Horizon 5 on November 8th, FSR 3 is a new version using AMD's MotionFluid technology to achieve double the fps of FSR 2, likely using a similar method to DLSS 3, further details are undisclosed as the feature is set to release in 2023. [9]

Nvidia Image Scaling (NIS)[edit]

Nvidia Image Scaling is an image scaling technique that uses directional scaling in four directions along with an adaptive sharpening filter, basically a spatial upscaler similar to what FidelityFX Super Resolution 1.0 does. There are two versions of NIS: a driver-level version that can be applied system-wide, and an SDK that can be used by developers to implement functionality on a per-program basis.

NIS was actually a part of the Nvidia Driver features since 2019 and the SDK was released for developers in November 2021. NIS didn't get much notoriety since it has only been implemented in a handful of games and its driver implementation is a bit confusing to use, although it can be used on any Nvidia GPU from Maxwell (GTX 745) and up unlike Radeon Super Resolution.[10]

MetalFX (MTLFX)[edit]

MetalFX (MTLFX) is an upscaling framework developed by Apple that integrates with their Metal graphics API and provides the ability to upscale a low-resolution image to a higher output resolution. The framework supports two different modes: a spatial upscaling mode akin to FSR 1.0, as well as a temporal antialiased upscaling mode similar to FSR 2.0.

The framework requires a Mac with the M1 chip or an Intel-based Mac running macOS 13 or later.[11]

Radeon Super Resolution (RSR)[edit]

An implementation of FSR 1.0 added into AMD drivers in Adrenalin 22.3.1, only usable on Radeon RX 5000 GPUs and above and require games with exclusive fullscreen. [12] RSR features enhanced support for borderless fullscreen games as of AMD Software Adrenalin 22.7.1.[13]

Temporal Anti-Aliasing Upsample (TAAU)[edit]

TAAU is an upscaling technique that combines TAA and spatial upscaling into one pass. This allows the technique to converge to sharper images than if the spacial upscaling was performed as a later separate pass. However, it is important to note that there are other teams using similar techniques, but with slight variations in the implementation. Unreal Engine's TAAU implementation has been superseded by TSR.

Temporal Super Resolution (TSR)[edit]

TSR is a upscaler similar to FSR 2.0 that requires motion vectors but does not require Machine learning Tensor cores, so therefore can theoretically be used by GPUs that still receive driver support. It has been made by Epic for their new Unreal Engine 5, but has also been backported to UE4. The quality of the image is said by Digital Foundry to be better then FSR1 but not as good as DLSS 2.3x.[14]

Xe Super Sampling (XeSS)[edit]

Xe Super Sampling is an image scaling technology developed by Intel for real-time use in select video games, using deep learning to upscale lower-resolution images to a higher resolution for display on higher-resolution computer monitors. Showcased multiple times in Intel's Architecture day and Arc Mobile announcements, XeSS will come in three options, one using XMX instructions which are accelated exclusively on Intel's own Arc GPUs, the other using DP4A instructions which are available in other GPUs from the AMD, Nvidia, and Intel, and the last is having support for Shader Model 6.4 which gives it the broadest support but the worst performance. [15] First XeSS game released was The DioField Chronicle on September 22nd, 2022, followed by Shadow of the Tomb Raider and Death Stranding: Director's Cut. Visual Quality is close to DLSS 2.x.x but lacking in certain areas like transparent and can produce moire patterns, performance improvement is mixed where weaker DP4A GPUs like the GTX 1060 and Shader Model 6.4 supported GPUs like AMD's pre-RDNA1 line-up don't see a benefit at all, meanwhile higher end gpus see performance similar to that of enabling DLSS, albeit a little lower. [16]

Supported games[edit]

For a list of games, see games with high-fidelity upscaling.

Notes

  1. although only a few games implemented it way after its release

References

  1. NVIDIA DLSS: Your Questions, Answered - last accessed on 2019-02-16
  2. NVIDIA DLSS 2.0: A Big Leap In AI Rendering - last accessed on 2020-03-27
  3. NVIDIA DLSS 3: AI-Powered Performance Multiplier Boosts Frame Rates By Up To 4X - last accessed on 2022-09-25
    Marvel's Spider-Man Remastered v1.1011.1.0 Patch Notes - last accessed on 2022-11-07
  4. Valve's Dota 2 Adds AMD FidelityFX Super Resolution - last accessed on 2021-06-24
  5. Announcing and First Look at AMD FidelityFX Super Resolution 2.0 - last accessed on 2022-03-17
  6. FidelityFX Super Resolution 2.0.1 (FSR 2.0) - last accessed on 2022-06-22
  7. God of War PC: AMD FSR 2.0 vs Nvidia DLSS Image/ Motion Quality Face-Off - last accessed on 2022-06-24
  8. It’s time to upscale FSR 2 even further: Meet FSR 2.1! - last accessed on 2022-09-09
  9. AMD previews FSR 3.0, which now includes frame generation - last accessed on 2022-11-07
  10. What is Nvidia Image Scaling? How to use the upscaling feature - last accessed on 2021-11-16
  11. Apple Developer Documentation - Applying temporal antialiasing and upscaling using MetalFX - last accessed on 2022-11-12
  12. AMD Software: Adrenalin Edition 22.3.1 Release Notes - last accessed on 2022-03-18
  13. AMD Software: Adrenalin Edition 22.7.1 Release Notes - last accessed on 2022-09-10
  14. Ghostwire: Tokyo on PC debuts impressive new DLSS competitor - last accessed on 2022-03-17
  15. Intel shows off its answer to Nvidia’s DLSS, coming to Arc GPUs in 2022 - last accessed on 2021-08-19A New Player has Entered the Game - last accessed on 2022-03-31
  16. Shadow of the Tomb Raider: XeSS vs. DLSS Comparison Review - last accessed on 2022-09-30Exclusive - Intel XeSS ML Upscaling - The Digital Foundry Tech Review - XeSS vs DLSS vs Native! - last accessed on 2022-09-30We Tested Intel's XeSS in Shadow of the Tomb Raider XeSS on Multiple GPUs - last accessed on 2022-09-30