steam: gamescope fails to launch when used from within steam

[hw-lunemann]

Describe the bug

gamescope crashes with this error when run from within steam's fhsenv:

wlserver: Running compositor on wayland display 'gamescope-0'
wlserver: [backend/headless/backend.c:18] Starting headless backend
wlserver: [xwayland/sockets.c:99] /tmp/.X11-unix not owned by root or us
wlserver: [xwayland/sockets.c:99] /tmp/.X11-unix not owned by root or us
[...]
wlserver: [xwayland/sockets.c:99] /tmp/.X11-unix not owned by root or us
wlserver: [xwayland/sockets.c:216] No display available in the first 33

This check in wlroots causes that error
https://gitlab.freedesktop.org/wlroots/wlroots/-/blob/0.15.1/xwayland/sockets.c#L98
because inside steam's fhsenv the /tmp/.X11-unix directory is owned by nobody:nogroup.
I have tried simply removing the check from wlroots but in that case gamescope simply hangs.
(Note that it's gamescope that depends on wlroots so this bug happens under x11 window managers too.)

Steps To Reproduce

Steps to reproduce the behavior:

1. start steam
2. add gamescope -- %command% to the launch options of any game
3. run the game

Notify maintainers

@jagajaga @jonringer

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.7, NixOS, 23.05 (Stoat), 23.05pre446447.1b1f50645af`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.12.0`
 - channels(root): `"nixos"`
 - channels(hadis): `"home-manager, nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[hw-lunemann]

This seems to be a very old issue with NixOS #42117

[hw-lunemann]

#215837 would allow gamescope to run with steam-run gamescope without having to manually chown /tmp/.X11-unix but running gamescope from within steam as described above is still broken.

gamescope dies with this error:

Xwayland: symbol lookup error: /nix/store/ip8l0dbilm0vayy5wqgghn05vsqp8130-libkrb5-1.20.1/lib/libcom_err.so.3: undefined symbol: k5_os_mutex_destroy, version krb5support_0_MIT
wlserver: [xwayland/server.c:247] Xwayland startup failed, not setting up xwm

However :

[nix-shell:/nix/store/ip8l0dbilm0vayy5wqgghn05vsqp8130-libkrb5-1.20.1/lib]$ readelf -Ws libcom_err.so.3 | awk '{print $8}' | grep k5_os_mutex_destroy
k5_os_mutex_destroy@krb5support_0_MIT
k5_os_mutex_destroy@krb5support_0_MIT

So that symbol is actually defined but Xwayland thinks it's not for some reason.

[GrabbenD]

I've encountered this issue too!
I know @jonringer got gamescope to run on NixOS - maybe we're missing something?

1. start steam
2. add gamescope -- %command% to the launch options of any game
3. run the game

/bin/sh\0-c\0gamescope -e -- /home/me/.local/share/Steam/ubuntu12_32/reaper SteamLaunch AppId=1240440 -- /home/me/.local/share/Steam/ubuntu12_32/steam-launch-wrapper -- '/home/me/.local/share/Steam/steamapps/common/SteamLinuxRuntime_sniper'/_v2-entry-point --verb=waitforexitandrun -- '/home/me/.local/share/Steam/steamapps/common/Proton - Experimental'/proton waitforexitandrun  '/home/me/.local/share/Steam/steamapps/common/Halo Infinite/HaloInfinite.exe'\0
Game process added : AppID 1240440 "gamescope -e -- /home/me/.local/share/Steam/ubuntu12_32/reaper SteamLaunch AppId=1240440 -- /home/me/.local/share/Steam/ubuntu12_32/steam-launch-wrapper -- '/home/me/.local/share/Steam/steamapps/common/SteamLinuxRuntime_sniper'/_v2-entry-point --verb=waitforexitandrun -- '/home/me/.local/share/Steam/steamapps/common/Proton - Experimental'/proton waitforexitandrun  '/home/me/.local/share/Steam/steamapps/common/Halo Infinite/HaloInfinite.exe'", ProcID 21934, IP 0.0.0.0:0
chdir /home/me/.local/share/Steam/steamapps/common/Halo Infinite
ERROR: ld.so: object '/home/me/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
GameAction [AppID 1240440, ActionID 1] : LaunchApp changed task to WaitingGameWindow with ""
ERROR: ld.so: object '/home/me/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
ERROR: ld.so: object '/home/me/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
wlserver: [backend/headless/backend.c:68] Creating headless backend
GameAction [AppID 1240440, ActionID 1] : LaunchApp changed task to Completed with ""
vulkan: selecting physical device 'AMD Radeon RX 6800 XT (RADV NAVI21)': queue family 1
vulkan: physical device supports DRM format modifiers
Steam: An X Error occurred
X Error of failed request:  BadWindow (invalid Window parameter)
Major opcode of failed request:  20 (X_GetProperty)
Resource id in failed request:  0x0
Serial number of failed request:  9
xerror_handler: X failed, continuing
Fossilize INFO: Overriding serialization path: "/home/me/.local/share/Steam/steamapps/shadercache/1240440/fozpipelinesv6/steamapprun_pipeline_cache".
vulkan: supported DRM formats for sampling usage:
vulkan:   AR24 (0x34325241)
vulkan:   XR24 (0x34325258)
vulkan:   AB24 (0x34324241)
vulkan:   XB24 (0x34324258)
vulkan:   NV12 (0x3231564E)
vulkan:   AB4H (0x48344241)
vulkan:   AB48 (0x38344241)
vulkan:   AB30 (0x30334241)
vulkan:   XB30 (0x30334258)
vulkan:   AR30 (0x30335241)
vulkan:   XR30 (0x30335258)
wlserver: Running compositor on wayland display 'gamescope-0'
wlserver: [backend/headless/backend.c:16] Starting headless backend
wlserver: [util/env.c:9] Loading WLR_NO_HARDWARE_CURSORS option: 1
wlserver: [types/output/output.c:382] WLR_NO_HARDWARE_CURSORS set, forcing software cursors
wlserver: [xwayland/server.c:108] Starting Xwayland on :1
ERROR: ld.so: object '/home/me/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
Xwayland: symbol lookup error: /nix/store/lkbxlllgchqavfl6ldz60vx0p7gkzc08-libkrb5-1.20.1/lib/libcom_err.so.3: undefined symbol: k5_os_mutex_destroy, version krb5support_0_MIT
wlserver: [xwayland/server.c:270] Xwayland startup failed, not setting up xwm

Workaround

See my comment here: #162562 (comment)

Related: #162562 (comment)

[ardishko]

Hi, I just wanted to real quickly add some context with my knowledge of how Steam works.

this part is not the issue and is an ""issue"" about the steam overlay, it happens because there is both a 32 bit and 64 bit version of the overlay both of which are loaded when you launch a 32bit or 64bit game, so this is not related. It happens every time you launch a game on any distro without fail since a game cannot both be 32 and 64 bit.
ERROR: ld.so: object '/home/me/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.

Everything else in this log seems normal, outside of this part:

Xwayland: symbol lookup error: /nix/store/lkbxlllgchqavfl6ldz60vx0p7gkzc08-libkrb5-1.20.1/lib/libcom_err.so.3: undefined symbol: k5_os_mutex_destroy, version krb5support_0_MIT which could point to a missing library. the mentioned "libcom_err.so" might be what we need to look at here.

Also, thank you so much to @GrabbenD for posting the workaround too, I was getting a headache dealing with this.

[urioTV]

Hi, i wanted to ask if someone is working on this issue?

[Octelly]

@urioTV It's an open-source project, so unless someone with the necessary knowledge is interested in the issue (could be yourself), then you'll probably have to wait.

[Electrostasy]

I ran into this same issue before. The only thing that let me run gamescope through steam so far has been the solution from this comment, but I am not sure why it works.

[Cryolitia]

I'm trying to add the patch in #162562 (comment) in #312268 , eagering for testing from you.

[K900]

There seem to be multiple issues conflated here. The original issue with X11 socket ownership has been addressed. If there's another issue that prevents running gamescope in Steam, please report it separately.