Support for high precision framebuffers and dither noise (RT64) (#252)

* 64 bit framebuffer for rt64

* Remove mention of motion blur cap from readme

* Add graphics.json option to control high precision framebuffers, disable clamping alpha for accumulation blur when using high precision FBs

* Increase dither noise strength at higher resolutions to make it more noticeable

---------

Co-authored-by: Mr-Wiseguy <mrwiseguyromhacking@gmail.com>

README.md

@@ -72,7 +72,7 @@ Saving and loading files, going from place to place, and pausing all happen in t
 #### Linux and Steam Deck Support
 A Linux binary is available for playing on most up-to-date distros, including on the Steam Deck.
 
-To play on Steam Deck, extract the Linux build onto your deck. Then, in desktop mode, right click the Zelda64Recompiled executable file and select "Add to Steam" as shown. From there, you can return to Gaming mode and configure the controls as needed. See the [Steam Deck gyro aim FAQ section](#how-do-i-set-up-gyro-aiming-on-steam-deck) for more detailed instructions.
+To play on Steam Deck, extract the Linux build onto your deck. Then, in desktop mode, right click the Zelda64Recompiled executable file and select "Add to Steam". From there, you can return to Gaming mode and configure the controls as needed. See the [Steam Deck gyro aim FAQ section](#how-do-i-set-up-gyro-aiming-on-steam-deck) for more detailed instructions.
 
 ## Planned Features
 * Dual analog control scheme (with analog camera)
@@ -108,7 +108,6 @@ You'll probably also want to change the default behavior so that you don't need
 If you want to play a modded ROM or in another language, note that support for modding and other languages will be added to the project itself in the future and will not rely on you supplying a different ROM. 
 
 ## Known Issues
-* The motion blur effect used by the game was capped to prevent ghosting at incredibly high framerates, which causes it to be less noticeable (this is only really noticeable above 120FPS). This may be fixed in the future by offering the option to render to an HDR framebuffer internally, which would allow it to be uncapped.
 * Intel GPUs on Linux may not currently work. If you have experience with Vulkan development on Linux, help here would be greatly appreciated!
 * The prebuilt Linux binary may not work correctly on some distributions of Linux. If you encounter such an issue, building the project locally yourself is recommended. A Flatpak or AppImage may be provided in the future to solve this issue. Adding the Linux version to Steam and setting "Steam Linux Runtime" as the compatibility tool or launching it via Gamescope may work around the issue. Alternatively, running the Windows version with Proton is known to work well and may also work around this issue.
 * Overlays such as MSI Afterburner and other software such as Wallpaper Engine can cause performance issues with this project that prevent the game from rendering correctly. Disabling such software is recommended.

include/rt64_layer.h

@@ -32,6 +32,7 @@ namespace ultramodern {
             void shutdown();
             void set_dummy_vi();
             uint32_t get_display_framerate();
+            float get_resolution_scale();
             void load_shader_cache(std::span<const char> cache_binary);
         private:
             RT64SetupResult setup_result;
@@ -40,6 +41,7 @@ namespace ultramodern {
     
     RT64::UserConfiguration::Antialiasing RT64MaxMSAA();
     bool RT64SamplePositionsSupported();
+    bool RT64HighPrecisionFBEnabled();
 }
 
 void set_rt64_hooks();

lib/rt64

@@ -1 +1 @@
-Subproject commit 1dd801264dbbf7a2f4e8fc483d28664852f50082
+Subproject commit 64b9e166f75b4dc44a59983b67c3e9ecc1f4cfd7

patches/effect_patches.c

@@ -98,11 +98,20 @@ void Play_DrawMotionBlur(PlayState* this) {
         f32 exponent = 20.0f / recomp_get_target_framerate(gFramerateDivisor);
         f32 alpha_float = recomp_powf(alpha / 255.0f, exponent);
         // Clamp the blur alpha, which ensures that the output color converges to within a reasonable delta of the target color
-        // when using an R8G8B8A8 framebuffer as RT64 currently does. Although this makes the effect less noticeable at high framerates,
+        // when using an R8G8B8A8 framebuffer. Although this makes the effect less noticeable at high framerates,
         // not clamping leads to noticeable image retention.
+        // Skip clamping if high precision framebuffers are in use, as there's no risk of ghosting with those.
+        if (!recomp_high_precision_fb_enabled()) {
             alpha_float = MIN(alpha_float, 0.825f);
+        }
         alpha = (s32)(alpha_float * 255.0f);
 
+        // @recomp Set the dither noise strength based on the resolution scale to make it easier to see at higher resolutions.
+        float res_scale = recomp_get_resolution_scale();
+        float dither_noise_strength = CLAMP(1.0 + (res_scale - 1.0f) / 8.0f, 1.0f, 2.0f);
+        // recomp_printf("res scale: %5.3f   dither noise strength: %5.3f\n", res_scale, dither_noise_strength);
+        gEXSetDitherNoiseStrength(OVERLAY_DISP++, dither_noise_strength);
+
         if (sMotionBlurStatus == MOTION_BLUR_PROCESS) {
             func_80170AE0(&this->pauseBgPreRender, &gfx, alpha);
         } else {

patches/graphics.h

@@ -5,5 +5,7 @@
 
 DECLARE_FUNC(float, recomp_get_aspect_ratio, float);
 DECLARE_FUNC(s32, recomp_get_target_framerate, s32);
+DECLARE_FUNC(s32, recomp_high_precision_fb_enabled);
+DECLARE_FUNC(float, recomp_get_resolution_scale);
 
 #endif

patches/syms.ld

@@ -52,3 +52,5 @@ osGetTime_recomp = 0x8F000088;
 recomp_autosave_enabled = 0x8F00008C;
 recomp_load_overlays = 0x8F000090;
 osInvalICache_recomp = 0x8F000094;
+recomp_high_precision_fb_enabled = 0x8F0000A8;
+recomp_get_resolution_scale = 0x8F0000AC;

src/game/config.cpp

@@ -24,6 +24,7 @@ constexpr auto api_default            = ultramodern::GraphicsApi::Auto;
 constexpr auto ar_default             = RT64::UserConfiguration::AspectRatio::Expand;
 constexpr auto msaa_default           = RT64::UserConfiguration::Antialiasing::MSAA2X;
 constexpr auto rr_default             = RT64::UserConfiguration::RefreshRate::Display;
+constexpr auto hpfb_default           = ultramodern::HighPrecisionFramebuffer::Auto;
 constexpr int ds_default              = 1;
 constexpr int rr_manual_default       = 60;
 constexpr bool developer_mode_default = false;
@@ -93,6 +94,7 @@ namespace ultramodern {
             {"ar_option",       config.ar_option},
             {"msaa_option",     config.msaa_option},
             {"rr_option",       config.rr_option},
+            {"hpfb_option",     config.hpfb_option},
             {"rr_manual_value", config.rr_manual_value},
             {"developer_mode",  config.developer_mode},
         };
@@ -107,6 +109,7 @@ namespace ultramodern {
         config.ar_option        = from_or_default(j, "ar_option",       ar_default);
         config.msaa_option      = from_or_default(j, "msaa_option",     msaa_default);
         config.rr_option        = from_or_default(j, "rr_option",       rr_default);
+        config.hpfb_option      = from_or_default(j, "hpfb_option",     hpfb_default);
         config.rr_manual_value  = from_or_default(j, "rr_manual_value", rr_manual_default);
         config.developer_mode   = from_or_default(j, "developer_mode",  developer_mode_default);
     }
@@ -245,6 +248,7 @@ void reset_graphics_options() {
     new_config.ar_option = ar_default;
     new_config.msaa_option = msaa_default;
     new_config.rr_option = rr_default;
+    new_config.hpfb_option = hpfb_default;
     new_config.rr_manual_value = rr_manual_default;
     new_config.developer_mode = developer_mode_default;
     ultramodern::set_graphics_config(new_config);

src/game/recomp_api.cpp

@@ -7,6 +7,7 @@
 #include "recomp_ui.h"
 #include "recomp_sound.h"
 #include "recomp_helpers.h"
+#include "rt64_layer.h"
 #include "../patches/input.h"
 #include "../patches/graphics.h"
 #include "../patches/sound.h"
@@ -101,3 +102,11 @@ extern "C" void recomp_load_overlays(uint8_t * rdram, recomp_context * ctx) {
 
     load_overlays(rom, ram, size);
 }
+
+extern "C" void recomp_high_precision_fb_enabled(uint8_t * rdram, recomp_context * ctx) {
+    _return(ctx, static_cast<s32>(ultramodern::RT64HighPrecisionFBEnabled()));
+}
+
+extern "C" void recomp_get_resolution_scale(uint8_t* rdram, recomp_context* ctx) {
+    _return(ctx, ultramodern::get_resolution_scale());
+}

ultramodern/config.hpp

@@ -27,6 +27,12 @@ namespace ultramodern {
 		Vulkan,
 		OptionCount
 	};
+	enum class HighPrecisionFramebuffer {
+		Auto,
+		On,
+		Off,
+		OptionCount
+	};
 
 	struct GraphicsConfig {
 		Resolution res_option;
@@ -36,6 +42,7 @@ namespace ultramodern {
 		RT64::UserConfiguration::AspectRatio ar_option;
 		RT64::UserConfiguration::Antialiasing msaa_option;
 		RT64::UserConfiguration::RefreshRate rr_option;
+		HighPrecisionFramebuffer hpfb_option;
 		int rr_manual_value;
 		int ds_option;
 		bool developer_mode;
@@ -68,6 +75,12 @@ namespace ultramodern {
 		{ultramodern::GraphicsApi::D3D12, "D3D12"},
 		{ultramodern::GraphicsApi::Vulkan, "Vulkan"},
 	});
+
+	NLOHMANN_JSON_SERIALIZE_ENUM(ultramodern::HighPrecisionFramebuffer, {
+		{ultramodern::HighPrecisionFramebuffer::Auto, "Auto"},
+		{ultramodern::HighPrecisionFramebuffer::On, "On"},
+		{ultramodern::HighPrecisionFramebuffer::Off, "Off"},
+	});
 };
 
 #endif

ultramodern/events.cpp

@@ -279,6 +279,7 @@ ultramodern::GraphicsConfig ultramodern::get_graphics_config() {
 }
 
 std::atomic_uint32_t display_refresh_rate = 60;
+std::atomic<float> resolution_scale = 1.0f;
 
 uint32_t ultramodern::get_target_framerate(uint32_t original) {
     ultramodern::GraphicsConfig graphics_config = ultramodern::get_graphics_config();
@@ -298,6 +299,10 @@ uint32_t ultramodern::get_display_refresh_rate() {
     return display_refresh_rate.load();
 }
 
+float ultramodern::get_resolution_scale() {
+    return resolution_scale.load();
+}
+
 void ultramodern::load_shader_cache(std::span<const char> cache_data) {
     events_context.action_queue.enqueue(LoadShaderCacheAction{cache_data});
 }
@@ -359,6 +364,7 @@ void gfx_thread_func(uint8_t* rdram, moodycamel::LightweightSemaphore* thread_re
                 events_context.vi.current_buffer = events_context.vi.next_buffer;
                 rt64.update_screen(swap_action->origin);
                 display_refresh_rate = rt64.get_display_framerate();
+                resolution_scale = rt64.get_resolution_scale();
             }
             else if (const auto* config_action = std::get_if<UpdateConfigAction>(&action)) {
                 ultramodern::GraphicsConfig new_config = cur_config;

ultramodern/rt64_layer.cpp

@@ -11,6 +11,7 @@ ultramodern::RT64Context::~RT64Context() = default;
 
 static RT64::UserConfiguration::Antialiasing device_max_msaa = RT64::UserConfiguration::Antialiasing::None;
 static bool sample_positions_supported = false;
+static bool high_precision_fb_enabled = false;
 
 static uint8_t DMEM[0x1000];
 static uint8_t IMEM[0x1000];
@@ -58,6 +59,19 @@ RT64::UserConfiguration::Antialiasing compute_max_supported_aa(RT64::RenderSampl
     return RT64::UserConfiguration::Antialiasing::None;
 }
 
+RT64::UserConfiguration::InternalColorFormat to_rt64(ultramodern::HighPrecisionFramebuffer option) {
+    switch (option) {
+        case ultramodern::HighPrecisionFramebuffer::Off:
+            return RT64::UserConfiguration::InternalColorFormat::Standard;
+        case ultramodern::HighPrecisionFramebuffer::On:
+            return RT64::UserConfiguration::InternalColorFormat::High;
+        case ultramodern::HighPrecisionFramebuffer::Auto:
+            return RT64::UserConfiguration::InternalColorFormat::Automatic;
+        default:
+            return RT64::UserConfiguration::InternalColorFormat::OptionCount;
+    }
+}
+
 void set_application_user_config(RT64::Application* application, const ultramodern::GraphicsConfig& config) {
     switch (config.res_option) {
         default:
@@ -95,6 +109,7 @@ void set_application_user_config(RT64::Application* application, const ultramode
     application->userConfig.antialiasing = config.msaa_option;
     application->userConfig.refreshRate = config.rr_option;
     application->userConfig.refreshRateTarget = config.rr_manual_value;
+    application->userConfig.internalColorFormat = to_rt64(config.hpfb_option);
 }
 
 ultramodern::RT64SetupResult map_setup_result(RT64::Application::SetupResult rt64_result) {
@@ -216,6 +231,8 @@ ultramodern::RT64Context::RT64Context(uint8_t* rdram, ultramodern::WindowHandle
         device_max_msaa = RT64::UserConfiguration::Antialiasing::None;
         sample_positions_supported = false;
     }
+
+    high_precision_fb_enabled = app->shaderLibrary->usesHDR;
 }
 
 void ultramodern::RT64Context::send_dl(const OSTask* task) {
@@ -261,6 +278,24 @@ uint32_t ultramodern::RT64Context::get_display_framerate() {
     return app->presentQueue->ext.sharedResources->swapChainRate;
 }
 
+float ultramodern::RT64Context::get_resolution_scale() {
+    constexpr int ReferenceHeight = 240;
+    switch (app->userConfig.resolution) {
+        case RT64::UserConfiguration::Resolution::WindowIntegerScale:
+            if (app->sharedQueueResources->swapChainHeight > 0) {
+                return std::max(float((app->sharedQueueResources->swapChainHeight + ReferenceHeight - 1) / ReferenceHeight), 1.0f);
+            }
+            else {
+                return 1.0f;
+            }
+        case RT64::UserConfiguration::Resolution::Manual:
+            return float(app->userConfig.resolutionMultiplier);
+        case RT64::UserConfiguration::Resolution::Original:
+        default:
+            return 1.0f;
+    }
+}
+
 void ultramodern::RT64Context::load_shader_cache(std::span<const char> cache_binary) {
     // TODO figure out how to avoid a copy here.
     std::istringstream cache_stream{std::string{cache_binary.data(), cache_binary.size()}};
@@ -278,3 +313,7 @@ RT64::UserConfiguration::Antialiasing ultramodern::RT64MaxMSAA() {
 bool ultramodern::RT64SamplePositionsSupported() {
     return sample_positions_supported;
 }
+
+bool ultramodern::RT64HighPrecisionFBEnabled() {
+    return high_precision_fb_enabled;
+}

ultramodern/ultramodern.hpp

@@ -111,9 +111,9 @@ void sleep_milliseconds(uint32_t millis);
 void sleep_until(const std::chrono::high_resolution_clock::time_point& time_point);
 
 // Graphics
-void get_window_size(uint32_t& width, uint32_t& height);
 uint32_t get_target_framerate(uint32_t original);
 uint32_t get_display_refresh_rate();
+float get_resolution_scale();
 void load_shader_cache(std::span<const char> cache_data);
 
 // Audio