Pipewire doesn't work

[luigir-it]

Bug report, debug log and your config file (FULL LOGS ARE MANDATORY)

https://pastebin.com/SDqmZr9f

Steps to reproduce

Open HyperHDR

What is expected?

Screen Grabbing

What is actually happening?

Screen Grabbing not happening (but if I select any effects they will work)

System

Arch Linux with Hyprland (Wayland)
I'm using the binary release

[awawa-dev]

Hi
A very briefly description and although there are logs, do I have to squeeze out further important information from you, how did you get that error (was there any system dioalog about capturing?) and in what configuration of your system;)? Please take the trouble and use the search first in discussions, or at least complete the relevant information about your system as I am not a clairvoyant. You can read about the limitations of the pipeware/portal itself, depending on e.g. distribution and it is not a HyperHDR problem, only some portal backends have been left behind or implemented the protocol 4 partially. It most likely works with the latest Ubuntu / Gnome release, and only it is guaranteed configuration as it stated in the HyperHDR release info. Other systems have to catch up with this, e.g. KDE, but Arch Linux also worked for me under Gnome. The logs show that either the implementation of the system window that asks the user to choose a screen was not implemented, restoration token was rejected (could happen if monitor disappears from the system for e.g. during sleep) or the user rejected the permission.

Suggest start from here: #333

[awawa-dev]

And I hope you did not try to run a HyperHDR from the command line or as a deamon because there is a warning about it...

[luigir-it]

I used the search function. I'm new to this system, I think I did all the necessary, like installing pipewire and xdg desktop portal wlr
Also, screen recording using obs does work. I tried to set the resolution to 1080p as you suggested to a user with a similar problem, but didn't work either.
I started the application simply by double clicking hyperHDR from the binary release.
I'm a noob, if you guide me through, i'll share all the info you need

Configuration: I have ws2812b to an arduino nano (115200 bad rate), the sketch i'm using is the following
https://pastebin.com/zYyWbikX

[awawa-dev]

I didn't test wlr because when the pipewire/portal was introducing protocol 4, wlr backend didn't implemented at this point. And it doesn't work under Virtual Box so I can't test it even now. The existing session portal token suggests that OS shown you at least once the dialog to select the screen to capture. Did the capturing work immediately after that? The token was stored and when you ran HyperHDR second? time it was rejected. Was the dialog shown again? Try to disable and enable software capturing in the HyperHDR grabber tab.

[luigir-it]

The capture never occurred.
I don't think the OS shown the dialog box, how can I make it prompt again?
I opened/closed the app and rebooted multiple time.
Already disabled and enabled back Software Capturing

[awawa-dev]

I don't think the OS shown the dialog box, how can I make it prompt again?

Disabling and enabling software capturing in the HyperHDR grabber tab should trigger that. But again check logs immediately after that because I'm not sure what you are clicking.

[awawa-dev]

You should not see something like that:

[PIPEWIRE_SYSTEM:PIPEWIRE SYSTE] Loading restoration token: ************************3256acfc9b8d

If you see it you didnt disable it....

[awawa-dev]

BTW OBS doesn't care about restoration token. But for HyperHDR is crucial to work: otherwise the user will be asked about the permission every time it runs HyperHDR. So even if OBS is working it doesnt mean anything at this point.

[luigir-it]

I switched to KDE Wayland and tried again.
Now after disabling and enabling screen compositor, the pop up showed up.
But the capture don't work unfortunately.
This is the new log:
https://pastebin.com/1RfkzjKS

As you can see, in the tray icon it says that something is recording, but leds are turned off

[luigir-it]

I tried again in KDE X11
The screen capture doesn't seem to work either
https://pastebin.com/taq6FxY1

[awawa-dev]

Neh, the capturing is working just fine.

[MUXER0] Register new input 'System/SYSTEMGRABBER' with priority 245 as inactive

But why? This priority is usually used by the USB grabber.

[awawa-dev]

I tried again in KDE X11

pipewire won't work for KDE/x11 they didn't implement that pipewire protocol

[luigir-it]

But why? This priority is usually used by the USB grabber.

What do I do about that? Should I change the priority?

[awawa-dev]

Nevermind, the USB grabber is disabled anyway. Priority 245 is active in your KDE/wayland logs. there is confirmed capturing around 10FPS and even in the live preview HYperHDR logo has disappeared: probably you are capturing the black screen that covered it. That symptom was described in the linked topic in the discussion. Please install Gnome ;)

[luigir-it]

Hello. I have an update.
Since last time I made various changes, and I finally managed to make it work!
So yes, if you configure it properly, it works even with the wlr backend of xdg desktop portal.

I don't know exactly what made it work, but here's what I've done:

1. Installed Hyprland with Nvidia and Screen Share patches: https://aur.archlinux.org/packages/hyprland-nvidia-git
2. I set the environmental variable as described here: https://wiki.hyprland.org/Nvidia/
3. I followed these instructions for screen capturing: https://gist.github.com/PowerBall253/2dea6ddf6974ba4e5d26c3139ffb7580
   Probably the most important thing was to remove the plasma implementation of the xdg desktop portal, as having both could cause problems.

[awawa-dev]

I don't see any problems with a pipewire in logs. Beside it probably doesn't support the restoration token which should work under protocol version 4 if the user selects it. Probably more can be seen when you run HyperHDR from the console. Currently only Gnome properly implement portal/pipewire for most configuration, most of the problems are due to fact that some of portal implementation (adding some problems with a pipewire e.g. missing x11 protocol on some configuration, capturing stream is being broken after the monitor is going to sleep) are incomplete or simply doesn't work properly. Currently I can only support Ubuntu/Gnome under VirtualBox because it was well tested and simply portal/pipewire is most stable here and I can reproduce a potential problem here. For other systems the proper target for reporting the issues is the portal backends the pipewire repo.

[luigir-it]

That's unfortunate but thank you for your support.
This is the console output, if it could be any useful: https://pastebin.com/t2B2DhEf

By the way, a "trick" that worked for me was this script, that I launch after starting the application:
while true; do sleep 60; curl -d '{"command":"componentstate","componentstate":{"component":"SYSTEMGRABBER","state":false}}' 'http://localhost:8090/json-rpc?request' > /dev/null ; curl -d '{"command":"componentstate","componentstate":{"component":"SYSTEMGRABBER","state":true}}' 'http://localhost:8090/json-rpc?request' > /dev/null ; sleep 1 ; ydotool click 0xC0 ; done

The annoying thing about this method is that when I disable the screen grabber, the leds turn off. Is there a way to not turn off the leds but instead leave them to the latest color? Before discovering this app and moving to Hyprland (wayland), I used the Psieg/Prismatik app, which didn't turn the leds off when closing the app. Thank you.

[awawa-dev]

I will try to test wlr on a physical PC and see if there is a workaround for this but it will take some time. From my perspective Wayland wasn't ready to replace x11 at this stage, it was "forced" decision and now the users pay for it. Nevertheless some popular distros and window system managed to implement somehow missing features so I'm focusing on them (like Ubuntu) hoping the rest will soon reach the same level of compatibility because at least the API is now universal.

The annoying thing about this method is that when I disable the screen grabber, the leds turn off. Is there a way to not turn off the leds but instead leave them to the latest color?

Only some specific LED drivers like Philips Hue and WLED (PR is waiting to be merge), support such feature (latest color before HyperHDR was turn on).

[awawa-dev]

I think I've got something, still can't reproduce your problem but I'm not using 4k resolution and I suspect it can be rather performance/resources issue that can be solved with a patch. Unfortunately we can't reduce or set constant pipewire framerate: stream's FPS can drop to zero when nothing happens, and jump to your monitor refresh rate when there are some changes on the screen, we can limit it only using our timers but the resources (waiting frames) are busy anyway so maybe you run out of free resources after some time. I'm finishing some changes that should enable for pipewire alternative & in the theory much faster capturing ONLY if the system supports it. It's not supported on virtual box so I suspect it works only with hardware devices. Maybe it can help also with your case. Are you willing to test that version on your system and provide the feedback?

[luigir-it]

Of course. I'm in my exam session so I may reply slowly in the next few days but I can test, no problem.

[awawa-dev]

Of course. I'm in my exam session so I may reply slowly in the next few days but I can test, no problem.

Great. No need to hurry, I need to make more testing.

As I wrote the capturing using pipewire/wayland is very tricky. When HyperHDR is running and there is almost no movement on the desktop top shows almost none CPU usage.

But make an experiment: run a terminal and from that terminal run top. So far everything is OK. But start moving that windows with the terminal using mouse very fast .... and now the things are starting to make ugly. In my case gnome-shell (in your case different composer of course) jumped to 500% (out of 800%) usage on 8-core CPU for 1080p60. If your are using 4k capturing probably it's even worse. That's the price of variable rate capturing which we can't control and some "benefits" from wayland.

That was the result for nouveau drivers running on physical PC with nvidia 1060GTX. Then I installed proprietary driver and CPU usage during rapid movements dropped to 70%. Still the driver doesn't offer DMA buffer for direct supports. But since the proprietary graphic driver was used, Ubuntu disabled wayland and fallback to x11/xshm. But HyperHDR's pipewire/portal grabber still worked, even better.

[luigir-it]

I tried it.
Doing stupid stuff with windows while using hyperhdr wasn't a resource intensive task:
CPU usage for Hyprland was at ~60% (out of 1200%)
GPU usage (monitored by nvidia-smi) was ~40% at max, using proprietary nvidia drivers with gtx1070

[awawa-dev]

So probably it's related to nouveau driver, that is missing an acceleration for the capturing. The same behavior is when I run it as typical emulated GFX under VBox.

When you tried OBS did you run it for a quite some time to see if it's stable? In OBS you just need to add 'pipewire/wayland screen capture' as a source and start the preview (you don't need to start the recording or streaming). Also running obs from the terminal command line should give more information about your system configuration.

[luigir-it]

For some reason that I don't understand, obs doesn't work anymore (when starting from terminal is says segmentation fault). It runs fine on plasma (wayland session) though.
I tried for ~6 minutes Wayfarer and it was "ok". Meaning that it recorded everything with occasional frame drops/stuttering.

[awawa-dev]

Not sure what can cause it. Yesterday I also tested latest&updated Fedora 36 image using physical PC, 1060GTX and KDE(Plasma). HyperHDR pipewire/capturing was working fine, even when starting the portal dialog asked if I want to remember the screen selection so portal protocol version 4 is fixed for KDE. On the other hand obs refused to worked with the wayland: the selection dialog also appeared but the preview was only the black screen. It didn't work for the entire screen nor for selected window capturing.

[luigir-it]

I tried again Wayfarer for ~17 minutes, this was the console output:
pipeline = queue name=queue0 ! videorate ! video/x-raw, framerate=30/1 ! videoconvert chroma-mode=GST_VIDEO_CHROMA_MODE_NONE dither=GST_VIDEO_DITHER_NONE matrix-mode=GST_VIDEO_MATRIX_MODE_OUTPUT_ONLY n-threads=7 ! queue ! vp8enc min_quantizer=10 max_quantizer=13 cpu-used=5 deadline=1000000 threads=7 ! queue ! matroskamux name=mux name=mux ! filesink name=filesink location="/home/luigi/Downloads/Wayfarer_2022-11-01_150623.mkv" pipewiresrc fd=33 path=81 do-timestamp=true keepalive-time=1000 resend-last=true ! video/x-raw, max-framerate=30/1 ! queue0. pulsesrc device="alsa_output.usb-iFi__by_AMR__iFi__by_AMR__HD_USB_Audio_0003-00.analog-stereo.monitor" ! opusenc ! queue ! mux. Gstreamer: EOS

[awawa-dev]

Hmm...I can't find any information here about the source (pipewire/wayland) if it's accelerated by using DMA or not. I think OBS may output such information.

[luigir-it]

I installed the flatpak version of obs and it did open, but the recording stopped after ~6/7 seconds.
This is the error:

And this is the terminal output:
obs_output.txt

[awawa-dev]

Seems a bit similar to this issue: obsproject/obs-studio#7534
What is interesting OBS doesnt support wayland on nvidia and that is not a good sign.

[awawa-dev]

I prepared an experimental version of HyperHDR: HyperHDR-19.0.0.0beta0-Linux-x86_64.tar.gz

Need more testing but on prehistoric Lenovo x220 i5 igfx running EndeavourOS (wayland/KDE), capturing works like a charm using the acceleration now. Almost no CPU usage. But Intel drivers are very good. I have second PC with gfx1060 and ubuntu/wayland: the acceleration is detected but pipewire refuses to use it and chooses the alternative methods. I've got locked secure boot here so I'm stuck with Ubuntu currently. You can find more information about the capturing process and if the acceleration works, in the output when you run the app in the console.

[luigir-it]

HyperHDR 19 doesn't start... It's probably due some library version issue, since Manjaro is always behind for updates.
error while loading shared libraries: libicudata.so.70: cannot open shared object file: No such file or directory
I have icu 72.1-1 and lib32-icu 72.1-1
Which version do I need? I'll try to upgrade/downgrade manually.
In a week or two I'm going to fresh install anyway

[awawa-dev]

You can copy it from the archive located the latest github action. I needed to trim that version because github allows only 25MB archive and that library is huge.

[awawa-dev]

Or probably the easy way should also work: I left libicudata.so.70 symlink in the archive lib folder and now it points to nowhere. Delete that symlink so your system libicudata should be used instead.

[luigir-it]

The library issue is fixed, HyperHDR now starts but it doesn't work: the recording preview is black

Meanwhile, after the system update, obs (non flatpak version) resumed working. So I recorded the screen for ~10 minutes and everything went fine (except audio that was very low quality, but I guess it doesn't matter). The recording was in 3440x1440, no quality issue and very smooth.

HyperHDR 18 does work (but with the usual issue)

Here's some logs
console-output.txt
hyperhdr-log.txt
obs-output.txt

[luigir-it]

Excuse me, but I don't understand why it is black, since the screen was not black when I saved the screenshot.

[awawa-dev]

I don't know. There was no error that would indicate something, everything was executed correctly so probably we simply received such image data for some reason. Intel:NVidia still 1:0.

[awawa-dev]

I have one more idea. From the archive lib folder try to delete:
libGL.so.1, libGL.so.1.7.0, libEGL.so.1, libEGL.so.1.1.0 , libGLX.so, libGLX.so.0, libGLX.so.0.0.0
to force your OS/NVidia libraries. Test it and for second try delete also
libGLdispatch.so.0, libGLdispatch.so.0.0.0
Maybe there is a conflict between GL/EGL distributed by HyperHDR and OS libraries. In that case we shouldn't distribute them.

[luigir-it]

Tried without luck
There's communication between the Arduino and HyperHDR, since if I change the luminance, the brightness of the leds change. But the color is always a light greenish, since it thinks that the screen is black.
I'll test again when I'll do a clean install of the OS, if you need more testing let me know.

[awawa-dev]

I need to investigate more nvidia case, but I don't like it because without errors hard to predict why the black screen is sent and everything points towards nvidia drivers. As I wrote I met such situation using obs #386 (reply in thread) contrary to obs, HyperHDR worked because at that time it didn't use the acceleration.

[luigir-it]

Hello.
Just wanted to let you know that I made a fresh install of Hyprland (wayland based on wlroots) and the same error occurs, with 19beta1. Everything work for ~2min and then stop (the leds having the colors of the last screenshot).
Meanwhile, OBS is working fine

[awawa-dev]

v19Beta1 doesn't have an acceleration for pipewire, it behaves like v18. Only the experimental build in this thread has it for testing purposes.

[luigir-it]

Ok, I just tried that version too, and there same problem is occurring (it's capturing a black screen) :\
Here's the log.txt

[awawa-dev]

Logs again looks OK, acceleration is detected, pipewire responds properly so I can rather exclude a bug here. I cannot reproduce the symptoms on my system, only met opposite situation when OBS was capturing black screen and HyperHDR worked. Since it's working also with other video graphic cards like intel I suspect it may be related to particular nvidia driver or Linux configuration/distribution. Maybe in next few weeks I will be able to have an access to similar configuration to verify it further.

[awawa-dev]

PR that introduces DMA & EGL Pipewire acceleration for HyperHDR is ready for testing #556

[awawa-dev]

Please attache full logs as this line is only a final 'goodbye' message. Github Action builds it's without a problem and you can download installer here:
https://github.com/awawa-dev/HyperHDR/actions/runs/4839534844

[luigir-it]

How can I download the installer? I don't see any download button... Maybe because "Publish Releases" is off?

[awawa-dev]

Sorry. It's on PR action it wont be uploaded. You must build it yourself in this case.

[luigir-it]

Output of failed build
cmake_out.txt
make_out.txt

[awawa-dev]

Your Qt6.5 is not supported. Add -DDO_NOT_USE_QT_VERSION_6_LIBS=ON -DENABLE_MQTT=OFF for cmake command.

[luigir-it]

Ok so I managed to build it and... It works!
Using the latest version (19), I had the problem described previously

Everything work for ~2min and then stop

I'm now been using the pipewire_hardware version for over 30 minutes and it works as expected.
CPU usage il also very low, ~0.5% on an old Ryzen1600

For some reason the web interface does not work, but I can access it and try new settings using HyperHDRv19

[luigir-it]

I don't know what MIME is, but the package shared-mime-info is installed

[awawa-dev]

Try to use browser console to debug why provided html content is not recognized by the browser as html (mime type in the content-type field), similar as in the linked issue.

[luigir-it]

With pipewire_hardware, Mime type is recognized as:
application/x-extension-html

With v19 it's text/html

[awawa-dev]

Probably something is not right with your development libraries (mixed qt5/qt6.5 on your system? or something else is missing) and QT mime types dont work for you. Unfortunately Qt doesn't tell us why. Arch Linux is supported since #520 and you can merge also this PR with #520 and let Github Action to build it for your when you commit it to your fork: Github Action must be enabled first. Or you can use clean docker https://github.com/awawa-dev/HyperHDR.dev.docker/pkgs/container/x86_64/85268877?tag=ArchLinux but it's more complicated.

[awawa-dev]

You can also try this https://bbs.archlinux.org/viewtopic.php?id=267553 Maybe it will help.