Automatically detect (some) non-steam games

[pinusc]

Thank you for making this amazing plugin!

While automatically detecting all non-steam games is obviously impossible, it should be possible to detect games added by certain managers such as lutris or bottles.

Bottles provides a CLI that can be used to extract paths of executables:

$ flatpak run --command=bottles-cli com.usebottles.bottles list bottles
Found 1 bottle:
- mygames
# then for each result
$ flatpak run --command=bottles-cli com.usebottles.bottles --json programs -b mygames
{"executable": "minit.exe", "arguments": null, "name": "minit", "path": "/run/media/mmcblk0p1/Games/minit/minit.exe", "folder": "/run/media/mmcblk0p1/Games/minit", "icon": "com.usebottles.bottles-program", "script": null, "dxvk": null, "vkd3d": null, "dxvk_nvapi": null, "fsr": null, "pulseaudio_latency": null, "virtual_desktop": null, "removed": null, "id": "7d401ec1-b261-40c0-a272-6d73df093c72"}]

Then steam games added via bottles (something that bottles itself does automagically) have target xdg-open and launch options e.g. bottles:run/'games'/'minit'

Lutris is even easier:

$ net.lutris.Lutris --list-games --json
[
  {
    "id": 15,
    "slug": "bloodlines",
    "name": "Bloodlines",
    "runner": "wine",
    "platform": "Windows",
    "year": null,
    "directory": "/run/media/mmcblk0p1/Games/Bloodlines",
    "playtime": "1:21:18.120709",
    "lastplayed": "2024-09-02 19:36:01"
  },
  {
    "id": 5,
    "slug": "celeste",
    "name": "Celeste",
    "runner": "linux",
    "platform": "Linux",
    "year": null,
    "directory": "/run/media/mmcblk0p1/Games/Bloodlines",
    "playtime": "0:00:12",
    "lastplayed": "2022-08-08 20:43:28"
  }]                                                    

The above information (both for lutris and bottles) doesn't conclusively identify which SD card a game is installed on, but MicroSDeck could in theory check for the existence of these files when run with a SD card inserted and do the rest.

[pinusc]

I should add that both Lutris and Bottles are among the easiest and, I think, most popular methods of installing non-steam games---which is to say, I don't think this is particularly niche, and a lot of users might benefit from this feature. That being said, I don't know how easy or hard it would be to implement, I just know it to be possible.

The other major installer/manager people use is heroic launcher. I have not used it, so this may be incorrect, but it looks like even though it does not provide a CLI like the above programs, it saves information about games in /home/deck/.var/app/com.heroicgameslauncher.hgl/config/heroic/sideload_apps/library.json, including game path - this is at least true for games added manually and not through e.g. logging in to Epic, but I'd expect a similar mechanism in place there.

[CEbbinghaus]

Thank you for your submission, First off I want to check which version of MicroSDeck you are using? In particular if you have the old version from the production store or are using the latest from testing / preview. It is possible to add any game from the Steam library to an SD card via the Edit MicroSD Card menu. This means no matter which manager you use, as long as the game is playable directly from steam (and not through the manager itself) it should all work the same.

Now if you are asking to be able to add games which do not exist within steam to the MicroSD card then I must ask why? At that point there is no game which can display which card it is on. The only way to view which card the game is on would be to flip through every single card and check the list of games for the existence of this one game.

The way that the database is set up is to use the steam appid as the primary key / link between the game & card. This means a game without an appid cannot exist. Assigning a random appid would run the risk of creating a collision with steam appids which although rare is currently an impossibility.

I recommend just adding all the games to steam as shortcuts (most if not all launchers support this. And there are third party tools to create the shortcuts if the launcher doesn't) and adding them to MicroSDeck that way.

[pinusc]

Thank you for the prompt reply! Sorry if my original issue description was not clear; I am aware that non-steam games can be manually added to microSDeck, I was outlining some steps as to how microSDeck might find and manage them automatically, just as it currently does with Steam games.

With Steam games, microSDeck does all the work of figuring out what games are installed on each SD card. There's no need to manually manage them.
With non-Steam games, currently microSDdeck presents the user with a list of them, and the user manually assigns them to SD cards. My proposal/feature request is that a (large) class of non-Steam games could be detected and managed automatically by microSDeck, requiring little to no user interaction - except checking the result and adding other non-Steam games as needed.

To clarify further, I am talking about non-steam games, in the steam library, and therefore with an assigned appid. The "detection" would simply be a mechanism that enables microSDeck to figure out where a non-steam game is installed, so that it would not be necessary for the user to manually tell microSDeck.

This might end up being a lot of effort, and possibly enough code to be out of scope for this project. However, it might also be reasonably easy, and I think it certainly would be a nice feature: some (a lot?) of users, myself included, mostly only add non-steam games from these launchers, so this feature would mean there is no need to manually manage games anymore... Manual management is great and I am grateful it exists, but it's a little annoying, and the sort of thing I consistently forget to do!

[CEbbinghaus]

Ohhh, I see what you are talking about. This is absolutely something I would consider. I think the biggest challenge to solve here is linking the output of those commands you provided with the games as they appear in Steam. Other than that I don't see any reason why this feature shouldn't exist.

Can you quickly confirm for me that running those commands will result in no output if no micosd card is inserted and some output if it is. Or does it always report the games it knows about and MicroSDeck would need to check if the file path exists?

Since I already have a lot of pressure to get MicroSDeck to 1.0 This will have to take a bit of a back seat. But I am on board with this functionality being added.

[pinusc]

Can you quickly confirm for me that running those commands will result in no output if no micosd card is inserted and some output if it is. Or does it always report the games it knows about and MicroSDeck would need to check if the file path exists?

Both lutris and bottles always reports the games, so the command output is the same no matter if the microsd is inserted or not.

I think the biggest challenge to solve here is linking the output of those commands you provided with the games as they appear in Steam. Other than that I don't see any reason why this feature shouldn't exist.

I have a possible flow for that, but figuring out a better/more elegant way might be challenging. Here's one approach:

1. MicroSDeck gets a list of non-steam games / appids
2. For each entry, read the target command + launch options.
   2.1 If it's just a path to an executable file, that's our, say, gamepath (simplest case, I haven't mentioned it previously but it should/could be handled)
   2.2 If it's a bottles command, get that bottle's name and then query bottles for the associated executable's path (using commands such as given above). If found, set gamepath to it.
   2.3 Similar for lutris
   2.4 For heroic, it might be necessary to parse configuration files manually as they don't provide a CLI/api AFAIK
3. At this point we should have found a gamepath that is, in essence, the executable file associated with a given appid.
4. Check if gamepath exists as a file in the currently plugged-in microSD
5. If so, associate the appid with this microSD
6. Profit

I am not very familiar with steam API/how to obtain the above information, and I haven't yet looked at microSDeck's code, so the viability of the above might vary. I might try to implement a proof of concept, but I'm not sure if I can submit a full pull request for this...

[pinusc]

After looking through MicroSDeck's code, I see that non-steam games are detected client-side in the UI, i.e. read from steam's own clientStore variable.

This complicates the above process a bit; however, non-steam games might also be obtained by the backend, by reading ~/.steam/steam/userdata/{userid}/config/shortcuts.vdf

This option is likely easier than the client-side sending all the found appids to the server for processing...

[CEbbinghaus]

I didn't end up finishing my response yesterday since I fell asleep. But your suggestions are all spot on. Yeah I was going to point out that its the fronted which adds the non steam games. I didn't even know about ~/.steam/steam/userdata/{userid}/config/shortcuts.vdf. That is an amazing hint and I will absolutely look into it. As long as we have a solid way to correlate the steam shortcuts with the cli outputs

[rkr87]

If it helps, Steam generates the shortcut app IDs using the appname and exe values.

class IDGenerator:
    """
    provides methods for converting various steam id formats to others...
    not sure why Steam has so many unique IDs for the same things...
    """
    @classmethod
    def generate_preliminary_id(cls, exe: str, appname: str) -> int:
        """
        all appids are based off this method,
        concatenates exe path and shortcut name
        """
        key: bytes = (exe + appname).encode('utf-8')
        top: int = int(hex(crc32(key)), 16) | int(0x80000000)
        return int(top) << int(32) | int(0x02000000)

    @classmethod
    def generate_app_id(cls, exe: str, appname: str) -> str:
        """used in big picture grids"""
        return str(cls.generate_preliminary_id(exe, appname))

    @classmethod
    def gen_short_appid(cls, exe: str, appname: str) -> str:
        """used for all other grids"""
        return str(cls.generate_preliminary_id(exe, appname) >> int(32))

    @classmethod
    def generate_shortcut_id(cls, exe: str, appname: str) -> int:
        """used as app_id in shortcuts.vdf"""
        prem_id: int = cls.generate_preliminary_id(exe, appname)
        return int((prem_id >> int(32)) - int(0x100000000))

I've been working on an app that automatically generates collections, those are the mechanisms steam uses for the various IDs it has (I'm not sure which one you can see).

[CEbbinghaus]

@rkr87 that is awesome. Thanks for the info. Although I kinda doubt that it would be very helpful. Will bookmark that bit of code for future. Here is hoping its considered a stable implementation and not something Steam changes just for fun.