steam-runtime-tools scripts may error with `getopt: command not found`

[R1kaB3rN]

Description

When running umu-launcher in certain environments/system configurations, the Steam Runtime tool's shell scripts may result in errors due to missing commands not found in PATH. In the case for v2-entry-point aka umu , I get the error getopt: command not found.

Terminal output

(deck@steamdeck ~)$ which umu-run
/home/deck/.nix-profile/bin/umu-run
(deck@steamdeck ~)$ PROTONPATH=GE-Proton9-26 umu-run ""
INFO: umu-launcher version 1.2.5 (3.12.9 (main, Feb  4 2025, 14:38:38) [GCC 14.2.1 20241116])
INFO: No GAMEID set, using umu-default
INFO: steamrt3 is up to date
/home/deck/.local/share/umu/umu: line 93: getopt: command not found
deck@steamdeck ~> cat .local/share/umu/VERSIONS.txt
#Name	Version		Runtime	Runtime_Version	Comment
depot	3.0.20250306.120299			# Overall version number
pressure-vessel	0.20250225.0	scout		# pressure-vessel-bin.tar.gz
scripts	0.20250225.0			# from steam-runtime-tools
sniper	3.0.20250306.120299	sniper	3.0.20250306.120299	# sniper_platform_3.0.20250306.120299/
deck@steamdeck ~/.l/s/umu [1]> rm umu-shim compatibilitytools.d.lock pfx.lock umu.lock
deck@steamdeck ~/.l/s/umu> mv umu _v2-entry-point
deck@steamdeck ~/.l/s/umu> ./pressure-vessel/bin/pv-verify --mtree mtree.txt.gz
pv-verify[20028]: N: Verified "/home/deck/.local/share/umu/sniper_platform_3.0.20250306.120299/files" against "/home/deck/.local/share/umu/sniper_platform_3.0.20250306.120299/usr-mtree.txt.gz" successfully
pv-verify[20028]: N: Verified "/home/deck/.local/share/umu" against "mtree.txt.gz" successfully

Reproduce

1. Setup home-manager (if possible, in a Steam Deck)
2. Install the umu-launcher NixOS package
3. Run PROTONPATH=GE-Proton umu-run ""

cc @MattSturgeon @LovingMelody @beh-10257 for insight

[MattSturgeon]

Steam Runtime tool's shell scripts may result in errors due to missing commands not found in PATH

The nixpkgs steam package bundles a number of dependencies in its FHS-environment: https://github.com/NixOS/nixpkgs/blob/6607cf789e541e7873d40d3a8f7815ea92204f32/pkgs/by-name/st/steam/package.nix#L22-L64

So far, the umu-lanucher package has held off adding many things to $PATH: https://github.com/NixOS/nixpkgs/blob/6607cf789e541e7873d40d3a8f7815ea92204f32/pkgs/by-name/um/umu-launcher/package.nix#L22-L29

(Note, however, that since NixOS/nixpkgs#381047 the umu-launcher does define the same profile environment, as we discovered this was needed in NixOS/nixpkgs#297662 (comment). We could use a similar strategy for runtime dependencies.)

I've not yet tried to reproduce the issue, however it sounds like the umu-launcher wrapper should be providing many (all?) of the same dependencies that the steam wrapper does in its FHS-env.

That said, we probably don't need all of them; some may relate to the steam launcher itself or running native games, so aren't relevant to umu, proton, or steamrt. If so, then we should just be able to add a few things like coreutils to the runtime dependencies. Is there a dependencies list anywhere?

[bohanubis]

#75
this is probably the issue
as far as I know its hardcoded to use the stuff in .local/share/umu no ??
for some reason when installed from home manager on other distros it doesn't work ??
anyway with more testing (context btw umu1 is the old local install or whatever you wanna call it umu is the new one)

at least on nixos there is no .local/share/umu/umu
why is that

[MattSturgeon]

#75
this is probably the issue

That issue sounds like it would affect nix-based installations, since anything copied from the nix store will be readonly.

However, "command not found" errors imply the PATH was searched. If you see my links above, you'll see that the umu-launcher wrapper package does not add many of the runtime dependencies that steam's wrapper package does.

I can see getopt in steam and heroic's FHS-env's $out/usr/bin directory, however not in umu-launcher's:

$ ls -l /nix/store/j4b9p9vm8g69v65rc2xb1y8qnbvhqvbj-steam-fhsenv-rootfs/usr/bin/getopt
lrwxrwxrwx 2 root root 67 Jan  1  1970 /nix/store/j4b9p9vm8g69v65rc2xb1y8qnbvhqvbj-steam-fhsenv-rootfs/usr/bin/getopt -> /nix/store/ki14bk4bhlrqnpkg4famkr1dnbda3ya9-getopt-1.1.6/bin/getopt*

$ ls -l /nix/store/vf8vzv9frq4f9rbpwks42hanl03vidjs-heroic-2.16.0-fhsenv-rootfs/usr/bin/getopt
lrwxrwxrwx 4 root root 76 Jan  1  1970 /nix/store/vf8vzv9frq4f9rbpwks42hanl03vidjs-heroic-2.16.0-fhsenv-rootfs/usr/bin/getopt -> /nix/store/8qp4dnhf02xg43c3yrqc63ivhxwhk950-util-linux-2.40.4-bin/bin/getopt*

$ ls -l /nix/store/lcr82knp6921j8spaaihxzw9ppcl7746-umu-launcher-1.2.5-fhsenv-rootfs/usr/bin/getopt
ls: cannot access '/nix/store/lcr82knp6921j8spaaihxzw9ppcl7746-umu-launcher-1.2.5-fhsenv-rootfs/usr/bin/getopt': No such file or directory

For steam, this is getopt 1.1.6, packaged as pkgs.getopt not the getopt from pkgs.unixtools (that would be 2.40.4).

I'm struggling to see where getopt gets added to the steam wrapper, as it doesn't seem to be part of any of the dependencies I can see listed.

For heroic, it is getopt from util-linux 2.40.4 and I can see util-linux being explicitly added to the wrapper's inputs.

[MattSturgeon]

1. Setup home-manager (if possible, in a Steam Deck)

2. Install the umu-launcher NixOS package

3. Run PROTONPATH=GE-Proton umu-run ""

Hm, I'm seemingly unable to reproduce the issue on NixOS (I can't test on SteamOS):

$ PROTONPATH=GE-Proton umu-run ""
INFO: umu-launcher version 1.2.5 (3.12.9 (main, Feb  4 2025, 14:38:38) [GCC 14.2.1 20241116])
INFO: No GAMEID set, using umu-default
INFO: Setting up Unified Launcher for Windows Games on Linux...
INFO: Downloading GE-Proton9-26.sha512sum...
INFO: Downloading GE-Proton9-26.tar.gz...
INFO: Downloading steamrt3 (latest), please wait...
INFO: SteamLinuxRuntime_sniper.tar.xz: SHA256 is OK
INFO: Verifying integrity of sniper_platform_3.0.20250306.120299...
INFO: sniper_platform_3.0.20250306.120299: mtree is OK
INFO: Using steamrt3 (latest)
INFO: GE-Proton9-26.tar.gz: SHA512 is OK
INFO: Extracting GE-Proton9-26.tar.gz...
INFO: /home/matt/.cache/umu/tmpk6e20x2w/GE-Proton9-26 -> /home/matt/.local/share/Steam/compatibilitytools.d
INFO: Using GE-Proton9-26
pressure-vessel-wrap[5748]: W: Found more than one possible libdrm data directory from provider
Proton: Upgrading prefix from None to GE-Proton9-26 (/home/matt/Games/umu/umu-default/)
ProtonFixes[5867] INFO: Running protonfixes
ProtonFixes[5867] INFO: Running checks
ProtonFixes[5867] INFO: All checks successful
ProtonFixes[5867] WARN: Game title not found in CSV
ProtonFixes[5867] INFO: Non-steam game UNKNOWN (umu-default)
ProtonFixes[5867] INFO: No store specified, using UMU database
ProtonFixes[5867] INFO: Using global defaults for UNKNOWN (umu-default)
ProtonFixes[5867] INFO: Non-steam game UNKNOWN (umu-default)
ProtonFixes[5867] INFO: No store specified, using UMU database
ProtonFixes[5867] INFO: No global protonfix found for UNKNOWN (umu-default)
Proton:
Proton: Executable is inside wine prefix, launching normally.
fsync: up and running.
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: File not found.

Command exited with status: 1

[MattSturgeon]

If you are able to reproduce with the flake in this repo, can you test if this patch helps?

diff --git a/packaging/nix/package.nix b/packaging/nix/package.nix
index 2030f87..b1572b5 100644
--- a/packaging/nix/package.nix
+++ b/packaging/nix/package.nix
@@ -41,5 +41,9 @@ in
         if overrides == {}
         then umu-launcher-unwrapped
         else umu-launcher-unwrapped.override overrides;
+
+      extraPkgs = pkgs: [
+        pkgs.util-linux
+      ];
     }
   )

Otherwise, if you're installing from nixpkgs you could use this overlay instead:

(final: prev: {
  umu-launcher = prev.umu-launcher.override {
    extraPkgs = pkgs: [ pkgs.util-linux ];
  };
})

E.g. using an overlay with home-manager

Assuming home-manager is managing its own pkgs instance (which is usually true, unless you have home-manager as a nixos module and enable home-manager.useGlobalPkgs), you can use home-manager's nixpkgs.overlays option to specify overlays:

# Either in home.nix or an imported home-manager module
{
  nixpkgs.overlays = [
    (final: prev: {
      umu-launcher = prev.umu-launcher.override {
        extraPkgs = pkgs: [
          pkgs.util-linux
        ];
      };
    })
  ];
}

[R1kaB3rN]

To note, I do not get the getopt: command not found error when using the umu-launcher-unwrapped package, and the command runs as expected:

deck@steamdeck ~ [1]> PROTONPATH=GE-Proton GAMEID=umu-0 umu-run ""
INFO: umu-launcher version 1.2.5 (3.12.9 (main, Feb  4 2025, 14:38:38) [GCC 14.2.1 20241116])
INFO: steamrt3 is up to date
INFO: GE-Proton is up to date
i386-linux-gnu-capsule-capture-libs: warning: Dependencies of /usr/lib32/libMangoHud.so not found, ignoring: Missing dependencies: Could not find "libxkbcommon.so.0" in LD_LIBRARY_PATH "", ld.so.cache, DT_RUNPATH or fallback /lib:/usr/lib
ProtonFixes[6565] INFO: Running protonfixes
ProtonFixes[6565] INFO: Running checks
ProtonFixes[6565] INFO: All checks successful
ProtonFixes[6565] WARN: Game title not found in CSV
ProtonFixes[6565] INFO: Non-steam game UNKNOWN (umu-0)
ProtonFixes[6565] INFO: No store specified, using UMU database
ProtonFixes[6565] INFO: Using global defaults for UNKNOWN (umu-0)
ProtonFixes[6565] INFO: Non-steam game UNKNOWN (umu-0)
ProtonFixes[6565] INFO: No store specified, using UMU database
ProtonFixes[6565] INFO: No global protonfix found for UNKNOWN (umu-0)
Proton: 
Proton: Executable is inside wine prefix, launching normally.
fsync: up and running.
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: File not found.

Command exited with status: 1
deck@steamdeck ~ [1]> which umu-run
/home/deck/.nix-profile/bin/umu-run

Otherwise, if you're installing from nixpkgs you could use this overlay instead:

Using the overlay where home-manager is managing its own pkgs instance, I get a new error:

INFO: umu-launcher version 1.2.5 (3.12.9 (main, Feb  4 2025, 14:38:38) [GCC 14.2.1 20241116])
INFO: Setting up Unified Launcher for Windows Games on Linux...
INFO: GE-Proton is up to date
INFO: Downloading steamrt3 (latest), please wait...
INFO: SteamLinuxRuntime_sniper.tar.xz: SHA256 is OK
INFO: Verifying integrity of sniper_platform_3.0.20250306.120299...
INFO: sniper_platform_3.0.20250306.120299: mtree is OK
INFO: Using steamrt3 (latest)
/usr/lib/pressure-vessel/from-host/libexec/steam-runtime-tools-0/pv-adverb: symbol lookup error: /lib/x86_64-linux-gnu/libpthread.so.0: undefined symbol: __libc_pthread_init, version GLIBC_PRIVATE

[R1kaB3rN]

This issue seems to be dependency related when entering the FHS-environment and only applicable to the official umu-launcher NixOS package. I haven't tested the Flake yet or to see if the issue is only specific to home-manager. Though, I don't expect the former to be an issue.

Hm, I'm seemingly unable to reproduce the issue on NixOS (I can't test on SteamOS):

That's strange, assuming you were using home-manager and umu-launcher.

[MattSturgeon]

To note, I do not get the getopt: command not found error when using the umu-launcher-unwrapped package, and the command runs as expected:

I guess because you're running steamos, all of steamrt's dependencies will be installed as part of the distro. So in that scenario it should be fine to use the unwrapped package.

The goal of the unwrapped package is to ensure all dependencies and env vars are correct regardless of that is installed on the main system.

It is strange/interesting that the wrapper package would make things worse though. Maybe bubblewrap blocks access to some of the host's environment?

Hm, I'm seemingly unable to reproduce the issue on NixOS (I can't test on SteamOS):

That's strange, assuming you were using home-manager and umu-launcher.

It shouldn't make a difference how the package is installed (nixos, home-manager, shell), whatever method is used it'll be the same package. It's more likely that something impure from the wider system is different. It's also possible that my system has even more dependencies missing than yours, and so I'm running into a different (earlier) issue.

I've tested both the nixpkgs package and this repo's flake. Both exit with this error:

Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: File not found.

(See full output above)

Using the overlay, I get a new error:

Looks like you may be using binaries that aren't nix compatible. Nix packages need to be built or patched to link correctly with nix-installed libraries. It could also be a lib version discrepancy. Hopefully once we get iron out all the required dependencies things will start to work 🤞

---

It's a shame we can't add a test to the nix package to cover this usage; nix tests need to be able to run in the build sandbox (without network access), so I don't think this case can be tested easily.

---

We can try adding more of steam's requirements to the overlay for now:

This overlay should bring in the same dependencies the steam wrapper does. I suspect many of them are steam client dependencies, not steamrt dependencies, but if it works with these we can try gradually commenting out the ones we don't think we need.

(final: prev: {
  umu-launcher = prev.umu-launcher.override {
    # https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/-/blob/main/docs/distro-assumptions.md#command-line-tools
    extraPkgs = pkgs: with pkgs; [
      # commented out, since it isn't listed in the steam package... maybe we'll need it anyway?
      # util-linux

      # We could also try using the other getopt package:
      # getopt

      # this shouldn't be needed, but maybe it has some transitives?
      # if uncommenting this helps, we can investigate further
      # steam-unwrapped

      bash
      coreutils
      file
      lsb-release  # not documented, called from Big Picture (we shouldn't need this)
      pciutils  # not documented, complains about lspci on startup (maybe we need this?)
      glibc_multi.bin
      xz
      zenity
    ];

    # https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/-/blob/main/docs/distro-assumptions.md#shared-libraries
    multiPkgs = pkgs: with pkgs; [
      glibc
      libxcrypt
      libGL

      libdrm
      libgbm
      udev
      libudev0-shim
      libva
      vulkan-loader

      networkmanager  # not documented, used for network status things in Big Picture
                      # (we shouldn't need this)
      libcap  # not documented, required by srt-bwrap
    ]
  };
})

[R1kaB3rN]

Additionally, I will say that the error getopt: command not found just started occurring all of a sudden. I used to be able to just run umu-launcher fine. Assuming the host isn’t related, it must of happened after updating the home-manager environment.

[MattSturgeon]

Assuming the host isn’t related, it must of happened after updating the home-manager environment.

The host env could be related.

Depending on what revision of nixpkgs you've updated from/to, a packaging regression is also possible. We could try git-bisecting your nixpkgs input if needed.

[R1kaB3rN]

This overlay should bring in the same dependencies the steam wrapper does

As far as dependencies goes, I believe we should only need to cover the ones between the boundaries of umu-launcher and the container runtime. In our case, since we only support container runtimes (e.g., soldier, sniper), this should only be the shell interpreter related or GNU coreutil related ones. The dependencies for the binaries within the pressure-vessel directory are already satisfied via RPATH.

[MattSturgeon]

Yeah, I just figured we'd start overkill with dependencies. If that works, then we can gradually remove some/most until things stop working again.

If having all the same dependencies as steam isn't enough, then there's probably additional issues unrelated to dependencies.

If this is a regression, then maybe something in the profile is causing issues. We recently changed the umu wrapper to use the same profile as steam's wrapper, although we probably don't need all of it.