asdf plugin spec gaps

[jdx]

these are just some places that I think the current asdf plugin model could be improved. This is a brainstorm of me getting most of my thoughts down—not designed to be a comprehensive plan. Admittedly, these are all ideas that are also very specific to rtx and its needs. Thinking about this from asdf's perspective and finding balance would obviously be important.

Also, it should go without saying but I don't think 100% of these should be done, more just ideas on what a "v2" of the model might contain.

version fetching/sorting standardization

working on rtx-versions I noticed the state of the world for version numbers is very poor. Many plugins return versions in the wrong order, or they only show the last "page" of github releases. GitHub releases generally suffers from rate limit problems with shared hardware. That's mitigated with the rtx-versions.jdx.dev host but I would like to improve this anyways.

github releases as a first-class concept

I think the majority (possibly large majority) of plugins use github releases. We should make that behavior more of a first-class citizen so we don't need to duplicate things like setting GITHUB_API_TOKEN in each plugin.

Boilerplate

The plugin template contains a lot of complex code like the following:

sort_versions() {
  sed 'h; s/[+-]/./g; s/.p\([[:digit:]]\)/.z\1/; s/$/.z/; G; s/\n/ /' |
    LC_ALL=C sort -t. -k 1,1 -k 2,2n -k 3,3n -k 4,4n -k 5,5n | awk '{print $2}'
}

I don't like that this gets duplicated in each plugin. I wonder if a setup where asdf/rtx provides a "standard toolkit" where things like this are already available would work better. Of course then we need to deal with versioning/maintaining that but I think it's a better world than just duplicating this everywhere.

too many files

I find reading plugins I'm often jumping from bin/download to bin/install to lib/utils.bash. Plugins never have that much code and I think just having a single file would make them easier to maintain.

In fact, I actually wonder if we could instead get by with a single toml file. Downside of that is syntax highlighting of bash inside of toml isn't likely to work very well. It wouldn't be as friendly for development since you can't execute them without asdf/rtx.

A bit of prior art here is asdf-java which uses symlinks to make all of the "bin/*" scripts the same file. Then that single file has a switch statement that does different things depending on which symlink called it. I think making this a standard in the plugin template would be a good way to simplify things and retain asdf compatibility.

plugin toml manifest

That said, right now there are a lot of moving parts with rtx for things like bin/list-bin-paths. Those are almost never dynamic but because it requires executing bash I cache the result in rtx. Caching hasn't been terrible but it's definitely something I would like to remove if I could. Since this is static data, I would prefer it if we could just have a toml file for this like the following:

list_bin_paths = ["bin", "foo"]

rtx actually supports this today (the exact syntax I might've remembered wrong) but it's not documented. Notably it's not that useful for asdf since they wouldn't have a good way to parse toml either.

releases/semver

I think plugins should work more like github actions where you can specify a major version of a plugin, e.g.:

rtx plugin install tuist@v1

That said, plugins really don't have a ton of development so there is fairly small utility. Still I think we could do better. Notably rtx itself doesn't use semver so I clearly don't think it's a fit for every use-case.

template uses conventional commits

I (personally) think conventional-commits only makes sense on teams where everyone is used to it. I think trying to get that to work for community developed plugins which require infrequent changes is a bad idea. Semver is also basically useless.

no support for bin/list-aliases

this is a feature specific to rtx for things like rtx install node@lts. It should be standardized.

bin/latest-stable should just be removed

asdf has a convention where the last version listed is the "latest" one. That makes this script unnecessary IMO. I think currently this script is actually recommended by asdf but that really confuses me (in fact it's the only script that isn't required they recommend using). I think we should advocate not including it.

There is some behavior around versions (in both asdf and rtx) that make this file necessary in very limited situations. That's for when the detection for what is/is not a "stable" version doesn't work (see code). I think this problem could be solved in a different way, possibly by giving plugins the ability to override this regex.

dependencies

we don't have any way to declare a plugin's dependencies (e.g.: python requires openssl). I think this is an obvious gap. I think we should stop short of trying to manage these dependencies, however. That's a job better handled by git.

bin/download is not fully thought-out

I think the idea that asdf/rtx need to have a download step and then a build step makes a lot of sense, but I think more should be done here. For example, I think checksum/gpg verification could be leveraged as well as a way to specify the hostname, setup mirrors, and store downloads could be improved.

bin/exec-env

This... is hard to summarize but it was really challenging to support in rtx. I don't have a suggestion right now on how to improve it, but I definitely don't like it very much.

One thought is if it outputted the env vars to stdout instead—that would make it a lot easier to deal with. That said, this hasn't been a source of maintenance burden but I imagine someone building a different front-end to these plugins will struggle here.

errors

I think error codes should be standardized. Maybe "code 100" could mean an http error or something. Dunno. Also don't know how helpful it would be to the end-user. Worth considering though, especially if there are ways we could improve error messages using these.

One idea is for the dependencies problem above. We could return "code 101" and emit to stdout something like the following: "MISSING: openssl"—then rtx could dress that error message up and show something like this for a macos/homebrew user:

$ rtx install ruby
[ERR] MISSING_DEPENDENCY_OPENSSL code: 1234
You must have openssl@[VERSION] installed on your system to install ruby.

Install openssl with:
  brew install openssl

scripts are either not used or rarely used

I think we could simplify the # of scripts generally. In fact I think a lot of things just aren't supported in rtx and nobody has ever complained since they're basically never used.

• bin/exec-path - I'm not sure if this is new or I've just never noticed it. At first glance I'm not even clear on what this is for or if it matters to rtx
• bin/post-plugin-add (and similar hooks) - these are almost never used

bin/help*

rtx doesn't use these at all. I like the idea of having help be integrated though. Potentially this could be displayed on a website or something. That said, there might be too many of these and we also might want to think about the structure and how they'd be consumed. In particular I don't like the idea of bin/help.deps and would prefer the plugin to just say "I need openssl" and asdf/rtx should figure out how to best communicate that to the user.

The downside here is that I looked at this before and it was hard to actually achieve so it's nontrivial. I also don't think these are used very heavily.

env vars

asdf/rtx use a lot of env vars like ASDF_INSTALL_VERSION to have asdf/rtx communicate with the plugin. I don't like that these have "asdf" since it doesn't work well for a (conceivably) rtx-only plugin. Or if we had a unified model.

I think coming up with a unified convention around these would be best. Like maybe TOOL_INSTALL_VERSION or something.

testing

This should be its own doc but I think it's important we improve testing/quality checks somehow. I think the current method of showing badges in the readme is clever but doesn't scale. Not sure what this would be but something could be helpful.

One thought is I'd like it if you could test them locally as well as in CI like brew test.

quality/security levels

We should somehow have some metadata/categorization so users know how much they can trust a plugin and what level of support it has. Related to this, we should have a concept of "first-party" plugins where the maintainers are the maintainers of the tool itself.

Alongside this they'll need a way to view this information in the CLI and in their browser.

[booniepepper]

The plugin spec from asdf is one of the things I like best from the project. It's minimal and easy to implement, and I think is a huge part of asdf and rtx success.

That said, I also think not only asdf/rtx are in this business of querying versions of tools and installing them. As an example, JetBrains IDEs also do this same kind of work and IntelliJ as a concrete example can manage runtimes and SDKs for Java, Kotlin, Groovy, Scala, and more. As far as I know, this is duplicated effort and not powered by existing version management software like sdkman/asdf/etc. There's also the interesting case of aqua which also provides tools at specified versions but operates only with static configuration files in its registry, rather than arbitrary shell scripts as plugins.

I know I'm the guy that wrote unconventions.org, but at some point I think it's worth moving this idea of specifying and formalizing the fetching of versions and assets of a project from a plugin spec to a more formal specification.

Where changes are needed, we can consider supporting the existing spec for asdf/rtx plugins. Where portability and security are a concern, we can recommend running the current shell-script-based plugins within simple containers.

[jdx]

That's a good point that there are plenty of reasons for wanting metadata about things like Java SDKs. We shouldn't limit ourselves to just version manager use-cases like asdf/rtx.

[jdx]

One thing I've always thought would be incredible is something like https://endoflife.date/ but that wouldn't need to be manually supported for each one somehow. Also changelogs and release notes. There might also be a way to tie this kind of thing into something like renovate.

[pepicrft]

I don't like that this gets duplicated in each plugin. I wonder if a setup where asdf/rtx provides a "standard toolkit" where things like this are already available would work better. Of course then we need to deal with versioning/maintaining that but I think it's a better world than just duplicating this everywhere.

Do you have anything in mind for how you would share a toolkit? Is this something that you can achieve by providing paths to scripts that they can source from their scripts?

I think plugins should work more like github actions where you can specify a major version of a plugin, e.g.:

Fun fact, Tuist has the concept of plugins, and users have already requested being able to specify a range of versions, but we are a bit on the fence with that one because then we need to introduce version pinning and commands for updating plugins.

In fact, I actually wonder if we could instead get by with a single toml file. Downside of that is syntax highlighting of bash inside of toml isn't likely to work very well. It wouldn't be as friendly for development since you can't execute them without asdf/rtx.

I didn't work too much with plugins, but when I had to implement Tuist's I didn't find it very inconvenient. The only thing that I personally find annoying, is having to write bash. But the world is so different now with Chat GPT. Have you considered using something like Deno or Bun? Developers would have better ergonomics, and you really don't care about the directory structure as long as they export functions following a conventional set of names. The downside is the dependency that you create on a runtime, which I don't think is a good idea for a tool of this nature. It's all trade-offs 😬

we don't have any way to declare a plugin's dependencies (e.g.: python requires openssl). I think this is an obvious gap. I think we should stop short of trying to manage these dependencies, however. That's a job better handled by git.

This would have the side effect of having a foundation to improve errors

I think the idea that asdf/rtx need to have a download step and then a build step makes a lot of sense, but I think more should be done here. For example, I think checksum/gpg verification could be leveraged as well as a way to specify the hostname, setup mirrors, and store downloads could be improved.

I'd consider separating download between "download" and "building" to that you can better cache those steps individually in the future. For example, if I install "X" in my system, and that requires "downloading" the sources and "building" them, you can skip "building" for the next developer installing the tool. I'd also require those phases to specify the inputs they depend on (files, environment variables), which you can then use for hashing purposes.

[jdx]

Do you have anything in mind for how you would share a toolkit? Is this something that you can achieve by providing paths to scripts that they can source from their scripts?

I'm thinking this would be baked into the rtx binary which would create bash files on first launch, or in the case of asdf just a part of the repo that it's expected you can import. This does mean they can't be versioned easily but I actually think that's fine. If needed, we could version individual functions like sort_versions_v2.

I didn't work too much with plugins, but when I had to implement Tuist's I didn't find it very inconvenient. The only thing that I personally find annoying, is having to write bash. But the world is so different now with Chat GPT. Have you considered using something like Deno or Bun? Developers would have better ergonomics, and you really don't care about the directory structure as long as they export functions following a conventional set of names. The downside is the dependency that you create on a runtime, which I don't think is a good idea for a tool of this nature. It's all trade-offs 😬

If we started looking at other languages I think what I would prefer is something like https://github.com/extism/extism that way people can use whatever language they want to write plugins in (cc @nilslice).

[nilslice]

would be happy to help if you're interested in getting this started!

[booniepepper]

One thing I really love about the implementation of aqua is how purely declarative @suzuki-shunsuke has managed to make its registry.

Here is me onboarding 4 small tools into aqua: https://github.com/aquaproj/aqua-registry/pull/16337/files

Note that the amount of code I have to care about per language is zero. These are just mappings of OS/Arch terms, and download URL patterns over time. Those might change, but change can be addressed by adding a new conditional URL pattern. There's no need for me to get into shell scripting (or any other language scripting) and writing out a if/then/else block etc

This works for a lot of tools: https://github.com/aquaproj/aqua-registry/tree/main/pkgs The only hole I've been able to find in this purely declarative strategy is that some tools require actions to compile or "set up" during installation. (See also: Erlang/OTP) Granted, it's a big hole, and people will want to compile some things. That is the part that should be a plugin, but there is a ton that can be simplified into way more static data like download URL patterns. Requiring scripts should be a fairly advanced use case. (And extism sounds like it might be fun for that.)

(I mostly wanted to distill the strategy and its observed success. Non-strategy details from aqua like the use of YAML for configuration, or renovate for version management, are irrelevant for arriving at an ideal specification here)

[pepicrft]

I like this idea. A declarative approach can be sufficient for most packages, and the scripting can serve as an escape hatch for plugins needing more. A declarative approach would lower the barrier for tool developers to provide an rtx plugin. When I created the one for Tuist and I got all the boilerplate scripting generated, which I barely modified, it felt very unnecessary tbh.

[reitzig]

+1 for exposing standard installation methods via non-code config.

GitHub release assets are certrainly a/the big one. A central, well-maintained implementation of that one would be awesome.

Regarding post-install scripts, I think some requirements analysis may be in order. One example that comes to mind is hooking into shell prompts (just like mise!) and adding shell completions (just like mise!). Basically, if I install a CLI tool (in user space), I also want to run its equivalent of tool completion fish > $HOME/.config/completions/tool.fish (if any). There may be enough structure there to manage this via TOML config as opposed to tailored shell commands.
(How shell completions should be added/loaded for project-specific environments, I don't have thoughts on atm.)

[reitzig]

It's worth noting that something like this would be an additive change: mise could keep support for asdf-style plugins while adding aqua-style plugins.