Add or update endpoint to support searching extensions by text

[jcfr]

Features

For a given set of app_revision, os and arch, user should be able to easily search extension by text.

Text should be looked up within the following metadata fields:

baseName
description

Questions

(a) How to implement autocomplete in web app backed by Girder 3 API endpoint ?

(b) Is there any existing implementation that could re-use or adapt from ? (ansible, config, end-point implementation, ....)

(c) Current plan is to experiment with option (5) and (3) described below.

Known MongoDB Limitations

• Using the built-in $text: {$search: 'sometext'} does not allow to implement search using arbitrary text as it is word based.

Options

In the context of the Slicer Extensions Manager migration where we are transitioning from my Midas to Girder for the backend, we have a Girder3 plugin called ^{1 2 3}.

The autocomplete capabilities associated with the Midas instance is implemented leveraging LIKE SQL operator. It allowed us to quickly display relevant extensions while the user was typing text in a search box.

See https://github.com/midasplatform/slicerpackages/blob/5c4c6cabd391609c57b4d2c4d06fc11658c8ebd5/models/pdo/ExtensionModel.php#L36-L73

This feature is important, and we have been exploring how to implement this using Girder 3.

Few options:
(1) Allocate funding for a MongoDB Atlas instance. See https://docs.atlas.mongodb.com/reference/atlas-search/autocomplete/ (btw $500 to $1000 / year)
(2) On premise install of Solr with use of mongodb connector ⁴.
(3) On premise install of Elasaticsearch with use of the mongdb connector "monstache" implemented in go ^{5 6}.
(4) On premise custom map/reduce job for extracting and updating collection to maintain map to original object⁷
(5) Simulate use of LIKE using regular expression⁸

References

Footnotes

1. https://slicer-packages.kitware.com/#collection/5f4474d0e1d8c75dfc70547e/folder/60ff9291266e4e7d3a596236 ↩

2. https://slicer-package-manager.readthedocs.io ↩

3. https://github.com/girder/slicer_package_manager ↩

4. https://blog.toadworld.com/2017/02/03/indexing-mongodb-data-in-apache-solr ↩

5. https://github.com/rwynn/monstache#readme ↩

6. https://rwynn.github.io/monstache-site/ ↩

7. https://stackoverflow.com/questions/29892947/implement-auto-complete-feature-using-mongodb-search/29903429#29903429 ↩

8. https://stackoverflow.com/questions/3305561/how-to-query-mongodb-with-like ↩