The packages in the "hash.tsv" file are not sorted alphabetically

[fabianegli]

I think it would be nice to have the package names sorted alphabetically in the hash.tsv file.

Why? Faster search for humans and the hash computation depends on the sorting, too.

[bgruening]

We can make that a recommendation for contributors and lint it, but is it worth it? Ultimately, you want probably a smarter search as a webapp. That could be easily built with a static site generator just reading all the files in the repo.

[fabianegli]

This is the idea. But why re-implement the hash generation? One place is enough. And I think this repo would be a natural single source of truth. Wouldn't it? I would say an auto-fix bot on the PR would be the right thing to do. Painless for contributors and maintainers (apart from setting it up).

[fabianegli]

(I am also not entirely sure if the sort in Javascript and Python are the same and for a static site I'd suggest using JS because pyodide still takes quite some time to load.)

[fabianegli]

I wonder a bit where the pip install galaxy-tool-util functions in the comes from because this seems to be archived and it is nowhere else in the galaxy project GitHub.

[bgruening]

Its here: https://github.com/galaxyproject/galaxy/tree/dev/packages/tool_util

So you would solely rely on GitHub search? Which is atm broken :)

[fabianegli]

🤣 Of course not, but I was dismayed by searching the galaxy repo and asking seemed reasonable. Also, the source code is not really where the link you posted points to but actually lives in https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/tool_util/.

This would probably warrant a different issue, but I think the homepage for galaxy-tool-util package on PyPI should point to a more specific location. But you see, I am not quite sure about this either, because the project structure might be set up in a way that this would also not be quite right, because even though we install "galaxy-tool-util" we get the package "galaxy".

All this to say it's (currently) not easy to find that code - if I hadn't spent more than a reasonable amount of time to search for it I wouldn't have asked here. And who would have thought that the search of GitHub is broken? 🤷‍♂️

[bgruening]

Also, the source code is not really where the link you posted points to but actually lives in https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/tool_util/.

This is the source, by link is the package that is then pushed to pypi. Those are links.

This would probably warrant a different issue, but I think the homepage for galaxy-tool-util package on PyPI should point to a more specific location.

Which one, its the galaxyproejct/galaxy repo. We could maybe point to the subdirectory? Not sure this is possible.

we get the package "galaxy".

What do you mean with package galaxy?

All this to say it's (currently) not easy to find that code - if I hadn't spent more than a reasonable amount of time to search for it I wouldn't have asked here. And who would have thought that the search of GitHub is broken?

Thats true. I have added a link to the readme.

Thanks @fabianegli for your feedback.

[fabianegli]

With pleasure :-)

Which one, its the galaxyproejct/galaxy repo. We could maybe point to the subdirectory? Not sure this is possible.

I believe the place to put it would be here: https://github.com/galaxyproject/galaxy/blob/c517e805771cc16807dfe675075a13fe6343f01f/pyproject.toml#L22

But that admittedly introduces cognitive dissonance, because the link would be called "repository" and pint to the inside of a repo. I believe it would work just fine, but I don't know.

[fabianegli]

Also, the source code is not really where the link you posted points to but actually lives in https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/tool_util/.

This is the source, by link is the package that is then pushed to pypi. Those are links.

True, the link I posted goes to the source, but from the link you posted (galaxy/tree/dev/packages/tool_util) I still had to click twice to find the relative reference in https://github.com/galaxyproject/galaxy/blob/dev/packages/tool_util/galaxy/tool_util and then follow that manually...

This is just an explanation why I said the link was not to the source code. And because I think this does not help the general discussion I'll hide it immediately as off topic ☮️

[fabianegli]

@bgruening I think for this whole thing it would be desirable to create a CLI in the spirit of what's already in the scripts folder. I imagine something that can be used in the following ways:

To validate:

lint_hash_tsv --check

To fix:

lint_hash_tsv --fix

This would allow easy use in continuous integration and autofix bots for GitHub PRs.

Any thoughts?

[jmchilton]

I think what would be more valuable than linting contributions would be a Github action that runs weekly or monthly and merges all contributions in that directory into hash.tsv and does the sorting at that time. I was originally imagining each contribution would come as its own file and some automated service would reduce it down into a single hash file to avoid the problem of too many files in a directory. Some projects do like release notes this way I believe and I think it seems like a good pattern for distributed collaboration.

[fabianegli]

Some projects do like release notes this way

Do you mean combined into one file, or in multiple files?

I think what would be more valuable than linting contributions would be a Github action that runs weekly or monthly and merges all contributions in that directory into hash.tsv

Some of the functionality required for linting and automatic fixing can be used for automatically merging in new mulled containers. And having a way to test the hash.tsv file for problems after automatically extending it is also a good idea, don't you think?

[mvdbeek]

Do you mean combined into one file, or in multiple files?

You grab content from many places and compile it down to a single document

And having a way to test the hash.tsv file for problems after automatically extending it is also a good idea, don't you think?

Seems fine, yes.