D3D11 Support Thread

[star69rem]

librashader-capi-tests doesn't even compile because it has hardcoded include paths to someone's D drive, and there only seems to be a Vulkan example. I just wanted to test out a D3D11 sample to see whether it was worth torpedoing one of my projects for.

[star69rem]

Yeah even if I manually add the Vulkan includes it doesn't compile.

Severity Code Description Project File Line Suppression State
Error (active) E0020 identifier "PFN_libra_preset_get_runtime_params" is undefined librashader-capi-tests E:\librashader-master\include\librashader_ld.h 295

[chyyran]

Hi! Sorry about that, I mostly focused on Rust examples when developing because that's what I'm familiar the most with

• Vulkan
• DirectX 11
• OpenGL

There was an issue with one of the type names regarding libra_preset_param_list_t that failed to get updated, I've since fixed that in e1eb0cc. That should hopefully fix the PFN_libra_preset_get_runtime_params error, but if it doesn't make sure librashader.h is also included. Another issue with the frame options naming for D3D11 was fixed in c201047. Please bear with me while these issues get sorted out.

As for the hardcoded "D:" drive includes, you'll have to fix those yourself in Visual Studio

Unfortunately I don't have a proper CMake setup for the C++ test project because I was mostly concerned with testing whether the bindings were sound or not rather than usage of the API which I had the Rust examples for. Here is a short snippet you can plug in to an existing D3D11 render loop with the latest commit.

libra_d3d11_filter_chain_t create_d3d11_filter_chain(libra_instance_t& instance, const char* path, ID3D11Device* device) {
    libra_shader_preset_t preset;
    libra_error_t error;

    error = instance.preset_create(path, &preset);
    if (error != NULL) {
        instance.error_print(error);
        instance.error_free(&error);
        return NULL;
    }

    libra_d3d11_filter_chain_t filter_chain;
    error = instance.d3d11_filter_chain_create(&preset, NULL, device,
                                               &filter_chain);
    if (error != NULL) {
        instance.error_print(error);
        instance.error_free(&error);
        return NULL;
    }

    return filter_chain;
}

void apply_d3d11_filter_chain_to_frame(
    libra_instance_t& instance, libra_d3d11_filter_chain_t& filter_chain, size_t frame_count, const ID3D11ShaderResourceView* srv, 
        const ID3D11RenderTargetView *rtv, uint32_t height, uint32_t width, float_t x, float_t y) {
    libra_source_image_d3d11_t input = { srv, height, width };
    libra_viewport_t vp = {x, y, height, width};

    // can check for error if you want 
    instance.d3d11_filter_chain_frame(&filter_chain, frame_count, input, vp,
                                      rtv, NULL, NULL);

}

Note that at this point in time, while in beta, I recommend you build your own copy of librashader.dll with the instructions in the README. There's quite a lot of churn since the last release but pushing a new release is blocking on some downstream dependencies to update.

[star69rem]

Thanks for looking into this. I'll try it out.

[star69rem]

No luck. Cannot get this thing to work as a static lib.

Severity Code Description Project File Line Suppression State
Error LNK2038 mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MT_StaticRelease' in librashader_capi.lib(wrapper.o) 1

The header doesn't work out of the box, either. It includes Vulkan stuff even if you're specifying D3D11.

[star69rem]

Seeing this when adding this to .cargo/config.toml:
[build]
rustflags = ["-C", "target-feature=+crt-static"]

libshaderc_sys-d34cdc21cb8ef69a.rlib(compiler.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MT_StaticRelease' in liblibrashader_spirv_cross-f96f5a6da73edc39.rlib(wrapper.o)

[star69rem]

And it doesn't work if you use librashader_ld.h, either, because that gets you:

Severity Code Description Project File Line Suppression State
Error (active) E0144 a value of type "libra_error_t (*)(libra_d3d11_filter_chain_t *chain, size_t frame_count, libra_source_image_d3d11_t image, libra_viewport_t viewport, const ID3D11RenderTargetView *out, const float *mvp, const frame_vk_opt_t *opt)" cannot be used to initialize an entity of type "PFN_libra_d3d11_filter_chain_frame" \librashader\librashader_ld.h 636

[star69rem]

Okay just for the heck of it, I tried letting it load the DLL. Surprise surprise, doesn't work either.

libra_instance_t librashader = librashader_load_instance();

libra_shader_preset_t preset;
libra_error_t error = librashader.preset_create("D:\\slang-shaders-master\\crt\\crt-royale.slangp", &preset);
if (error != NULL) {
	std::cout << "Could not load preset\n";
	
}

filter_chain_d3d11_opt_t shaderOptions = {};
shaderOptions.use_deferred_context = true;
shaderOptions.force_no_mipmaps = false;

libra_d3d11_filter_chain_t chain;
if (librashader.d3d11_filter_chain_create(&preset, &shaderOptions, m_d3dDevice.Get(), &chain)) {
	std::cout << "Could not create OpenGL filter chain\n";
}

Bombs out when falling chain_create.

Exception thrown at 0x00007FFA438ECB0E (librashader.dll) in .exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

[chyyran]

And it doesn't work if you use librashader_ld.h, either, because that gets you:

Severity Code Description Project File Line Suppression State Error (active) E0144 a value of type "libra_error_t (*)(libra_d3d11_filter_chain_t *chain, size_t frame_count, libra_source_image_d3d11_t image, libra_viewport_t viewport, const ID3D11RenderTargetView *out, const float *mvp, const frame_vk_opt_t *opt)" cannot be used to initialize an entity of type "PFN_libra_d3d11_filter_chain_frame" \librashader\librashader_ld.h 636

Should be fixed in 62b0d59. Forgot to apply type renaming fixes to the loader header, since that one is manually written and not generated.

To link as a static lib, you shouldn't need to add rustflags = ["-C", "target-feature=+crt-static"]. Instead you link against librashader.lib, and you'll have to add that yourself in Visual Studio. This will pull in a bunch of system linkages that you'll need to resolve as well depending on your project.

librashader_ld.h will enable support for all supported runtimes by default. You should remove the LIBRA_RUNTIME_VULKAN define to disable Vulkan support.

[chyyran]

Bombs out when falling chain_create.

Exception thrown at 0x00007FFA438ECB0E (librashader.dll) in .exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

Are you using the fixed header in 62b0d59 for this? Can you try passing in NULL for the shader options. I'll take a look later today if neither of those work out.

[star69rem]

error is 0x000001d8a1958750 when I do it with NULL, but it isn't outright crashing anymore.

libra_error_t error = librashader.d3d11_filter_chain_create(&preset, NULL, g_game->m_d3dDevice1.Get(), &chain);

[star69rem]

And libra_d3d11_filter_chain_t chain is 0xcccccccccccccccc.

[star69rem]

To link as a static lib, you shouldn't need to add rustflags = ["-C", "target-feature=+crt-static"]. Instead you link against librashader.lib, and you'll have to add that yourself in Visual Studio. This will pull in a bunch of system linkages that you'll need to resolve as well depending on your project.

That's what I did, and I get the mismatch errors.

error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MT_StaticRelease'

My project is statically linked, but that .lib is not regardless of whether I mess with the rust flags.

[chyyran]

Thanks, I can look more into this after work.

…

On Thu, Jan 26, 2023 at 10:31 AM star69rem ***@***.***> wrote: To link as a static lib, you shouldn't need to add rustflags = ["-C", "target-feature=+crt-static"]. Instead you link against librashader.lib, and you'll have to add that yourself in Visual Studio. This will pull in a bunch of system linkages that you'll need to resolve as well depending on your project. That's what I did, and I get the mismatch errors. error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MT_StaticRelease' My project is statically linked, but that .lib is not regardless of whether I mess with the rust flags. — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHUIN3HBLDBI7A433J5TIDWUKKGBANCNFSM6AAAAAAUCAO7TQ> . You are receiving this because you commented.Message ID: ***@***.***>

[star69rem]

I just checked, and the only reason it didn't crash with the access violation was when it was called before the d3d device was created. Also note that this is a ComPtr, and I'm getting the pointer with ComPtr Get() if that matters.

[chyyran]

I suspect that it has to do with the library expecting a pointer to the ComPtr. I’ll have to fix the generated headers to account for that.

…

On Thu, Jan 26, 2023 at 11:07 AM star69rem ***@***.***> wrote: I just checked, and the only reason it didn't crash with the access violation was when it was called before the d3d device was created. Also note that this is a ComPtr, and I'm getting the pointer with ComPtr Get() if that matters. — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHUINZ5MERGYQCVY7KIVBTWUKOMDANCNFSM6AAAAAAUCAO7TQ> . You are receiving this because you commented.Message ID: ***@***.***>

[chyyran]

Those errors show up with Lottes on RetroArch as well if you have the debug layer enabled. It's a problem with the shader, not the runtime.

[star69rem]

fyi zfastcrt just gives me a black screen. All libra_error_ts are NULL.

[chyyran]

zfast-crt works fine for me. I don't see anything out of the ordinary in RenderDoc.

[star69rem]

There's definitely something going on for me because it seems like a third of the shaders just give me a black screen.

[chyyran]

I would just plug it into RenderDoc and see where the image gets lost. The dx11 runtime unbinds all render targets after frame so maybe something needs to be rebound.

[chyyran]

I doubt its a driver issue because I test on a 3070.

[star69rem]

Do some of the shaders require a depth stencil view? I'm not creating one.

[chyyran]

As far as I understand RetroArch does not supply a depth stencil view to shaders.

[star69rem]

Kind of at a loss then why a lot of the shaders just output a black screen for me.

[star69rem]

I am on the very latest Nvidia driver btw. The one that came out on 1/24.

[chyyran]

You can use RenderDoc to make sure that no SRVs are unbound during the shader passes and that the pipeline state is correct after shader processing.

[star69rem]

If that was the case, though, wouldn't every shader fail? If something was being unbound, how would CRT Royale work?

[star69rem]

This is scalefx\scalefx-9x.slangp in Retroarch:

Here it is in Librashader:

[star69rem]

So something changed and I can't even compile Librashader anymore. Can't test it until I work out whatever broke.

[chyyran]

The D3D12 runtime added a dependency on spirv-to-dxil, so you'll need the following Python packages to build.

pip install meson ninja mako

[star69rem]

Not it.
C:\msys64\librashader>pip install meson ninja mako
Requirement already satisfied: meson in c:\msys64\mingw64\lib\python3.10\site-packages (1.0.0)
Requirement already satisfied: ninja in c:\msys64\mingw64\lib\python3.10\site-packages (1.11.1)
Requirement already satisfied: mako in c:\msys64\mingw64\lib\python3.10\site-packages (1.2.4)
Requirement already satisfied: MarkupSafe>=0.9.2 in c:\msys64\mingw64\lib\python3.10\site-packages (from mako) (2.1.1)

Compiling spirv-to-dxil-sys v0.3.0
error: failed to run custom build command for spirv-to-dxil-sys v0.3.0

[star69rem]

Maybe it has something to do with these undefined cmake variables.

CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-msvc = None
CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_msvc = None
HOST_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_x86_64-pc-windows-msvc = None
CMAKE_GENERATOR_x86_64_pc_windows_msvc = None
HOST_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
CMAKE_PREFIX_PATH_x86_64-pc-windows-msvc = None
CMAKE_PREFIX_PATH_x86_64_pc_windows_msvc = None
HOST_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_x86_64-pc-windows-msvc = None
CMAKE_x86_64_pc_windows_msvc = None
HOST_CMAKE = None
CMAKE = None

[chyyran]

Are you building this on the MinGW toolchain as opposed to MSVC?

[star69rem]

zfast capture.

https://archive.org/details/zfast-crt-capture1

[chyyran]

Thanks, I'll take a look once I get time after I wrap up the D3D12 backend.

[chyyran]

One thing you can test real quick is if you are driving the shader by providing frame_count. Not all shaders require this but some do, it has to be a monotonically increasing size_t per frame.

[chyyran]

This looks like an issue with the blend state not being set properly. It's working for me before because the example renders straight on top the swapchain image but if it's a blank texture it looks to render on top of well, nothing.

Hopefully I've fixed that in 5ffcf00. Let me know if you still have problems with zfast after this commit.

[star69rem]

I am.

[star69rem]

I did verify that our scalefx-9x.slangp files are identical. I never modified them.

[chyyran]

Is the output of libra_preset_print the same as #2 (reply in thread) modulo file paths?

[chyyran]

Re: crashing issue

The first thing I can think of that could potentially cause driver crashes on initialization is parallel shader compilation introduced in

f5fe3e3

if you created your D3D11 device with D3D11_CREATE_DEVICE_SINGLETHREADED

You can disable multithreading there by setting the RAYON_NUM_THREADS=1 environment variable and seeing if that fixes the crashes. If so I can add a check for this to fall back to single threaded shader compilation if the device was created as single threaded.

[star69rem]

Yup, that's what it was. I was using D3D11_CREATE_DEVICE_SINGLETHREADED. Works with creation flags set to 0.

[chyyran]

07b20fe will disable multithreaded shader compile if the device was created as D3D11_CREATE_DEVICE_SINGLETHREADED

[star69rem]

Scalefx seems to be working now too. During fade transitions I see what look like little artifacts but when the image is stable it looks stable. That might just be how it behaves.

[star69rem]

Just wanted to let you know I'm really happy with how Librashader is working now, and performance is pretty good.

For comparison, Final Fight in Retroarch with CRT Royale uses ~10% of my GPU @ 60hz. At 4:3 with an internal resolution of 384x224 with CRT Royale through Librashader, my program uses 20% GPU at 144hz. Running CPS games at ultrawide resolutions (692x224) uses about 30% GPU usage at 144hz, which is still playable.

I've got an option integrated into the in-game menu for it.

https://cdn.discordapp.com/attachments/1011419989246488598/1073147787480420352/image.png

[chyyran]

Thanks, glad everything works out.

[chyyran]

Just FYI, I'm looking to release 0.1.0 soon, committing to a stable API for the 0.1.x semver series. There's been a couple of ABI-breaking changes since last week, particularly with option structs to allow for forward evolution without breaking ABI further, and the new D3D11 frame API taking an optional ID3D11DeviceContext to allow for finer control of deferred contexts if desired.

Since you're probably the only person currently using librashader in a project, if there are any features you would like to add that could potentially require an API break, now is the best time for that sort of feedback before I cut a non-suffixed numbered release.

[star69rem]

For frame count, what happens if I let the size_t overflow? If I don't handle it with a comparison to size_max and reset it to 0, will things blow up?

[chyyran]

The library can’t control what bit pattern it gets for frame_count. If you’re calling it from C or C++ where unsigned overflow is UB, then yes, things will blow up if you don’t check beforehand. If you’re calling from a language where overflow is well defined, it follows the semantics of that language. frame will just pass whatever bit pattern it gets (if it gets called at all) to the shader uniforms.

Realistically speaking, SIZE_MAX is an unfathomably large number so the risk of it actually overflowing is pretty low.

[chyyran]

Any internal state doesn’t actually rely on the frame count that’s passed in though, for example the Vulkan runtime keeps an internal frame counter for frames-in-flight scheduling that explicitly wraps but that’s not observable to the caller nor any shaders.

The D3D11 runtime doesn't have anything like that though.

[star69rem]

Something else I noticed that might not be a bug, but I'm just letting you know in case something is going on.

I have an options screen with a very dark background logo:

With CRT Lottes, I can still make out the background:

With CRT Royale, it looks like it gets completely crushed to black. Is that just the shader, I wonder, or a possible bug?

You should be able to test the first raw 384x224 to verify.

[chyyran]

I checked in RetroArch and it looks the same to me.

[star69rem]

Nevermind the post about the build not working. The last VS2022 upgrade broke my Rust environment. Yay!

[star69rem]

I know you can't possibly be expected to support this scenario, but using the latest RC, Librashader stopped working for a couple of my users who were using a Windows build through Wine. Older versions (that predate CRT Royale working) seemed to work for them in Wine.

[chyyran]

Does it work via Proton?

[star69rem]

Am being told it doesn't work with proton-ge or wine-ge.

[chyyran]

It could be because Wine is trying to load symbols from D3D12. You can try building librashader-capi with only D3D11 support by changing the default features from runtime-all to runtime-d3d11 in Cargo.toml.

[star69rem]

Yes, I rebuilt with runtime-d3d11, and it works for that person. Thanks!

[star69rem]

More reports from users, and apparently it was even breaking in Windows 11 for some people, and the build with runtime-d3d11 fixed it for them as well. So it really isn't just a Wine issue.

[chyyran]

I did some local testing and believe the issue to be the import of dxcompiler.dll which might not be available on some systems without a D3D12 shader compiler previously installed. I didn't run into this issue because the Vulkan SDK automatically installs DXC as a system library.

I did some local testing and the https://github.com/SnowflakePowered/librashader/tree/fix-delayload-dxcompiler branch should fix it without needing to build runtime-d3d11. It's probably unreasonable for me to ask your users to do testing for me but some confirmation that this fixes the issue would be nice to have.

[star69rem]

I did a build of that branch and it works for the Wine user.

[star69rem]

Can you try ./assets/shaders/cubic/b-spline-fast.slangp?

I'm not seeing the same output in my program as Retroarch.

[chyyran]

Fixed in #13

[chyyran]

Actually scratch that, I was a little too hasty and it didn’t compile so gonna need to fix that in a little bit.

[chyyran]

0.1.3 should fix this properly. Was just an oversight from cleaning up sampler set creation