gpu-next: cscale x2 filter like super xbr #12163
Comments
|
Generally these kinds of scalers are best left as user shaders. I don't want to derail this FR into a discussion, but I would be curious about seeing some real content examples. I currently use My general impression of super-xBR for luma scaling is very poor, so I don't see how it would outperform a bilateral scaler for chroma, which uses the "ground truth" as a reference for scaling the chroma plane. This is not even mentioning the fact that bilateral scalers perform even better when you're using a good luma prescaler to increase the resolution of the luma information. If you're unsatisfied with the results produced by KrigBilateral and/or if you think the performance impact is too high, I recommend playing around with the JointBilateral and FastBilateral user shaders. If I remember correctly, KrigBilateral scales chroma by mapping the correlation between the chroma and luma planes (hence "Krig"Bilateral). This allows KrigBilateral to produce values that are outside of the original chroma range, which might be undesirable for some users. Joint and FastBilateral don't do this, so they might produce more satisfying results for you? If you think I'm completely off-base, then feel free to correct me. |
|
if the chromaloc issue wouldn't be there a user shader wouldn't be a problem. it's literally not possible to use super xbr without a x scale that fixes the chroma placement. loading a different shader depending on chromaloc is just not realistic. chroma scaler matter most where no luma is to guide that's why bilateral scaling is overrated in my experience. https://drive.google.com/file/d/12YUZbkUqkCGHcU25Ri_bCV0P5PaliDDU/view?usp=sharing this test has perfect luma guide and biliteral will not even win here. madVR: bilinear for control: super xBR: bilateral sharp: reconstruction placebo AR: mpv settings: scale-antiring=0.8 # luma upscale deringing glsl-shaders-clr #dither=no bilateral: cscale=ewa_lanczossharp scale-antiring=0.8 # luma upscale deringing (this is used for chroma too isn't it?(gpu-next)) and this is an case where bilateral should have shinned... no source to reproduce yet because it is an utter pain to create such: the most difference is right shoulder. it's not even funny how super xbr totally destroys them all here. that's the beauty about it it is consistently not doing bad like biliteral here and can win hands down. i can do much much more but i would like to avoid this if possible because it is a lot of work with chroma and i have done it to much over the past 10 years. |
|
Quick question - what is "super xBR"? I tried to search for it on mpv wiki and I did not find it. |
|
it doesn't have it. it does extremely well as a chroma scaler and with a proper AR setting is usual "better" than jinc as an image scaler while also been cheaper but limited to x2 scales. |
It's a scaler designed for video game emulators to "smooth" out pixel art. It's in the same vein as scalers like SuperEagle, HQ2x, 2xSaI etc. I personally find this family of scalers extremely ugly for emulation, but apparently there are people that like them... It tends to produce results that are oil paint-like, and can be aliasy.
OK, so for some historical context (any developers/contributors lurking this thread are free to correct me), Super-xBR used to be a built-in prescaler in mpv years ago, along with NNEDI3. This was changed because maintaining the code for these prescalers was a chore for development, so they were ported as user shaders instead. You can track the discussion for this decision in these two threads (the first thread is similar to @mightyhuhn's FR): #3226 #3236 After a few years, bjin stopped generating Super-xBR in his repository due to the general consensus around Super-xBR being very negative. You can still build Super-xBR yourself using this script contained in the source branch, or you can browse older snapshots of the repository and download As for NNEDI3 and RAVU, bjin stopped generating the However, as with ... Now, for the actual test images themselves, I managed to track down a high quality source for my own testing. My comparisons aren't of the same frame, so it's a little hard to reproduce this 1:1 (I also don't use or have madVR installed on my system). A few things I've noted during my testing:
Here's a link to my own comparison: https://slow.pics/c/SJx8pvr0
For superxbr screenshot:
Fast/JointBilateral were left at the default tune. tl;dr, I personally don't see why Super-xBR needs to be implemented back into mpv. Obviously I'm not a developer, and if the dev team wants to implement Super-xBR back into mpv, then that's their decision. But from everything I've gathered so far, I don't think the devs want to re-implement built-in prescalers. |
this time upscaled to 4K because the difference is staggering: if there would be a save way to get an x scale move to correct for chromaloc that also takes half sub pixel moves of scaler into account i wouldn't even dare to make this feature request. |
What I meant is that it's an edge-case, and isn't representative of 99% of content. There aren't a lot of instances where you're meant to pay attention to borderless red serif font on a pitch black background. This red text sequence in the OP lasts exactly 2.043 seconds out of a 1m32s OP, and out of a ~24 minute-long episode.
It means the source has been post-processed, and we're moving further away from the ground truth. It's hard to trust scaling results when they could be the fault of a bad studio upscale. A quick glance at anibin shows that the HDTV broadcast for this show was mastered in 1280x720, and even the author acknowledges the jaggies caused by the aliasing. I think it's safe to assume that the BD is just an upscale of this native resolution, but again, I'm too lazy to do a descale. In the grand scheme of things, it probably doesn't matter.
The source of this image appears to be the official website. However, it's a lossy encode of what seems to be an artifacted source, and I'm just going to make the assumption that the original artist didn't intend to have banding or mosquito noise in their artwork... I don't have madVR, so it's kind of hard to recreate any of the results here since I don't even know if Bilateral in madVR is equivalent to any of the bilateral user shaders I've described so far, so I'll generate my own test image and lay out some clear reproduction steps to be as transparent as possible. Credit goes out to @Artoriuz and his meme blog, as I don't know any better methodology of subsampling chroma. I tried to pick a source image that would be as favorable as possible towards Super-xBR. So this means something with a lot of flat colors and minimal gradients. To keep up with the Gundam theme... I went with this fanart of Sazabi since it's my favorite suit. It doesn't have subsampled chroma, and it's also very red, which should make this test easier. I did the following to generate a lossless reference that I could screenshot with mpv: And to subsample the image: I took screenshots with mpv using the same scalers as my previous comparison. However, this time I also added a luma upscale with RAVU-Zoom-AR. This should increase the difference between the bilateral filters and the rest of the scalers (and arguably, this is the only use-case where bilateral filters aren't placebo...). Link: https://slow.pics/c/rP1KHTen My conclusions still remain the same. This isn't even mentioning the fact that you don't necessarily need to use ... Either way, this subjective discussion has been dragging out for a while, but like I said, the chroma variants of NNEDI3, super-xBR and RAVU are still accessible through the repo history. Instead of dragging out the discussion any longer, it might be better to just have @bjin chime in on why these user shaders were deprecated (even though I feel like the answer is fairly obvious just by looking at previous discussions and commit comments). |
All these 2x chroma prescalers are deprecated because they can't replace Super-xBR has long been deprecated because it's only meant for lowres pixel art footage (e.g. retro 8-bit games), and brings artifacts like aliasing and ringings. With the introduce of ravu, which is faster and has much less noticeable artifacts, I see no points to use Super-xBR anymore. I later removed |
FWIW, madVR is not using anything special. Their Bilateral is also from @Shiandow (sorry for summoning you). For legal reason all HLSL sources are available in Point being that if you really want you can port those (over 8 years old) HLSL madVR shaders to mpv and have the same look and feel. |
i just took stuff i find fast i can find chroma all over the place pretty much everywhere.
if i play an upscaled blu ray i play an upscaled blu ray it's doesn't matter the chroma scaler has to do it job.
the super xbr results are terrible because it is the only image that is moved or moved to far even creating new colors chroma chroma bleeding the scaler in mpv is not representive.
thank you the first line hits the nail and the reason for this feature request x2 scaler need guide/x scale based on the source file chromaloc. ravu chroma left is not good enough. the original use case of super xBR does not matter to me it does not perform well there or maybe madshi changed so much it's a dead end anyway then super xbr is dead but there still other scaler i corrently can'T even try because well the reason of this feature request. i'm personally not willing to pay more than ewa lanczos or super xBR for chroma but that's my personal problem andfor madVR super xbr was cheaper. just to point it out again just look at the right shoulder of the character that alone is enough reason for me to never use it again. https://slow.pics/c/7iTtfshW i personal never use test pattern i recommend real content and base a scaler on that. if scaled, noisy or what ever the scaler has to deal with that. scaler like NGU sharp are pointless to me because they need a prestine source... the madVR stuff is literally the mpdn stuff for other that care: krig should be the old bilateral or maybe sharp can't tell the mpv version shown here and tested by me is different not going to call it worse/better on such a low number of tests: |
I don't disagree... I'm just saying that it's hard to come to an informed conclusion without a better source. If the source has artifacting, it's hard to tell how much of the artifacting is attributed to the source, and how much of it is attributed to the scaler.
I'm not quite sure what you mean here, but yes, when you subsample the chroma, the chroma plane isn't going to "tightly" fit onto the luma plane anymore. I guess you could call that blurriness a type of chroma "bleed", but there isn't any mixing of colors here. The chroma just isn't perfectly mapped onto the pixel intensities anymore. You have some of the chroma smearing into the blacks, and some grey intensities that are not covered by the chroma anymore. Super-xBR is never going to be as "sharp" as a bilateral filter, so obviously it's going look more akin to a traditional scaler when applied to chroma. If you feel like my methodology flawed, feel free to reproduce it, as I've given the steps to do so. But none of these results feel unexpected to me. Funnily enough, I associate color blending or "mixing" with KrigBilateral, not traditional scaling kernels. Because like I said earlier in the thread, KrigBilateral is allowed to generate chroma values that are outside of the given chroma range. For example, this can sometimes mess up stuff such as chromatic aberrations (but to be fair, I think most bilateral filters will fail a chromatic aberration test for reasons outside of just color mixing). Just be clear, I don't think bilateral filters are perfect! If I did, I would be using them instead of |
I've tried doing something like this and, as far as I remember, it doesn't work. Maybe it could work with the AR ravu-r2 (the one without negative weights), but it probably wouldn't work with normal versions of ravu. Filters with a negative second lobe are generally sharper, but these weights need nearby pixels to be different in magnitude in order to sharpen. However, with the cross/joint scheme you want to reduce the contribution of pixels with distant luminosities, which are usually also the pixels with distant chromaticities... These 2 things end up working against each other because the negative weights from the spatial filter will be greatly reduced after they get multiplied by the weights from the intensity filter, making the output less sharp. It also means your negative weights can, sometimes, outweigh the positive ones depending on how "unlucky" you are with luminosity, which obviously creates a bunch of artifacts. |
|
here is your red mecha. still green lining from bleed so still bad test. there is also a 1080p upscale to compare mpv ravu superxbr to superxbr (even for image) bicubic 150 AR. look at the cockpit entrance there are huge black stair in the ravu image and ringing so much ringing... there is also no need to focus so hard on super xbr. the point of the feature request is still "2x" cscale chromaloc corrected scaler in an a user friendly matter. |
|
Would something like this be good enough for your use-case?
If yes, just use this. I haven't been able to fully test this yet, and generally speaking I think chroma shaders are a waste of time, but if it solves your problem feel free to use it. This image was generated with the 4-tap variant of the shader + |
|
If you're considering just using a linear regression you're probably better off using Kriging. You'll be susceptible to the same kinds of artifacts but at least you have some control over them. For completeness, here is that same image scaled with KrigBilateral: Anyway, I'm just here because I kept getting pinged. For what it's worth my 2cts is that the simplest solution it to add the chromaloc to the shaders as a parameter, don't see how you're going to get user shaders for chroma to work properly otherwise. |
|
@Shiandow Thanks for you work on MPDN rendering scripts and improving the state of art bar of HQ video rendering in general!
I just want to add that mpv shader is now able to fix chroma offset and align chroma and luma planes with a header attribute |
the now make it sound to me that this is new? what so ever looks like the direction want this to go.
something is wrong with your image you have a green tint in it. i don't want to represent your work wrong. but i'm not aware of a glsl version for mpv of bilateral directly from you so i have to take what i find. the superxbr version used by dokur0chan is hardly the one in mpdn too https://github.com/zachsaw/MPDN_Extensions/blob/master/Extensions/RenderScripts/Super-xBR/super-xbr.hlsl |
Hmm, not sure where that came from, I can confirm it in your image but not in a screenshot I take locally. I'm blaming github for that one. Not sure how I can attach an image in a way that prevents that from happening. I can give it another go, but no promises: I think KrigBilateral is what Madshi renamed into BilateralSharp. Apparently adding the name of a South African mathematician that nobody knows how to pronounce is 'confusing'. I wouldn't be able to tell you which version of the script runs where at this point, the one I use looks like the one in the MPDN repository. Though I can't rule out that I tweaked some parameters at some point. I'm pretty sure Madshi made some changes of his own to prevent it from going off the rails, but it's surprisingly hard to predict when things go wrong. Or to be precise it's very hard to tell the difference between it correctly filling in some small hard to see details and it incorrectly imagining those details out of noise. It's a very clear example of the bias/variance trade-off. |
|
yeah that's why i don't like it. it's very hit and miss. when it hits it hits hard but when it misses it can miss really really badly: the only reason i even bring it up is that in the mpv community it's often just stated use krigbilateral and move on and i strongly disagree or would at least be able to test other stuff. it is old, soft and sharp in madVR calling one of them krig would be odd wouldn't it be? |
request for integrated cscaler like super xbr or ravu.
there is currently krigbiliteral as an "high end" chroma external shader but that's a very hit and miss scaler and very undesirable for me.
because of the nature of chromaloc and some scaler as shader are an unpleasant experience.
example:
nnedi3 moves the image by half a sub pixel so this is fine for UHD chromaloc source if done in the correct way but will be wrong for mpeg2 chromaloc meaning an x scaler shader should be called after it. making a set and forget shader not possible atleast AFAIK.
other scaler like super xbr are always centred meaning they need correction depending on the video chromaloc information. having them been handled directly would make it easier for user to change to x2 chroma scaler and a scaler like super xbr should be in the same ballpark as jinc in term of performance cost.
real content examples why these scaler can matter can be provided. i personally doubt anything over super xbr is ever worth the performance impact but i'm glad been proven wrong.
The text was updated successfully, but these errors were encountered: