Unusual Reponses from Filter Query

[jmcguir]

Hi everyone,

I've ran into a weird one that is kind of preventing us from filtering devices (or prefixes/vm's) by site in the UI. All of our sites are numbers and letters, generally less then five characters. For example we have a site named 85 and 85A. We have around 1000 sites following that format. The issue comes in when you try to filter by site(when using the filter tab on a primary object). When I look up 85 it returns nothing. The specific query it is using is this (found via the inspector in chrome (F12)):

https://dev.netbox.test.com/api/dcim/sites/?brief=true&q=85

The problem is that this returns 833 results and the site called 85 isn't in the first 50 (it paginates and only looks at the first 50).

So I guess I have two questions:

1. is there a reasonable workaround?
2. Why is this query returning so many results. Looking through the first 50 I only see one that has "85" in the brief output and that was as a substring in the object ID.

Let me know if I can provide more info.

Thanks so much!

Joel McGuire

[mtinberg]

q= is a text search and is casting a wider net than what you want, you might query instead by name= to get a more exact match. Look at the API docs in Swagger to see the full list of query parameters each API endpoint supports /api/docs/ as there are more options than is obvious if you are using the UI queries as your guide.

ah you mentioned the filter tab, so this is related to queries through the web interface not the api and is related to netbox 3.x.

I attempted to replicate and was not able, if I go to demo.netbox.dev then Devices and the Filter tab and in the Sites filter add "Athens" the query looks is https://demo.netbox.dev/dcim/devices/?q=&site_id=43

[jmcguir]

Thanks!

The problem is that Netbox itself is generating these queries via the UI. So I can't really filter by site in this case at all inside the UI.

Here is a screenshot of what is happening:

FWIW our zip code starts with 85 which could be a source of issues. We just upgraded yesterday to 3.0.8

[jmcguir]

I don't think this will be able to be replicated on the demo site because there are not 50+ sites so it won't hit the limit.

[jeremystretch]

ah you mentioned the filter tab, so this is related to queries through the web interface not the api

FYI both use the same filter sets.

[mtinberg]

Oh, I'm getting a clue now, the field does type-ahead-find and searches using the API to give you a filtered list of options as you type, you can see that a new query is issued with each keystroke and the resulting list should be filtered to just Sites that match the query, and the end you can scroll down and pick one to have that specific site added to the filter.

The problem you are having is that the query is limited by the default pagination of 50 and you have enough sites that match the queries for short strings that the specific site you need to pick from the scroll-able list is not on it, you can't type any more text to filter further and the type-ahead-find doesn't run through all the paginated results because why would it, you wouldn't want that widget populated with potentially thousands of items.

Manually adding the search criteria to the URL by Site slug will work but is obviously not as good as using the filter widget.

https://demo.netbox.dev/dcim/devices/?site=dm-akron

You might be able to change the default PAGINATE_COUNT setting from 50 to something larger, but that is going to have some performance penalty, I'm not sure you want to scroll down a list of 1000 items every time you type in a search widget. I'm not sure how the type-ahead-find widget works (jQuery, something else?) and if it has options to take an exact search term and not present the option list at all, the only thing I can think of is that you can put the search terms in the URL by hand (which is what the filter widget accomplishes)

[jmcguir]

Thanks for the time to write it all out. You put it much better than I did initially :). Thinking about this overnight, I guess my concern is twofold:

1. If I am searching for site why is it searching through the physical address at all? That is the only thing I think of that could be matching for a search of 85 in our sites (because most of our sites have a zip starting with '85'). I would think the default query would look for just names and slugs in this form (at least for site but likely for all). I find a pattern of looking for all the sites in Tucson by typing Tucson into the filter field to be too clever by half for example. It also doesn't work because only show's sites that actually match the pattern even if they have Tucson in their address since it filters after the query for sites that match the query string in that first 50.

Screenshot of behavior of looking for sites in Tucson by looking for Tucson:

2. I could try and downgrade our instance but I don't remember this behavior in 2.x.x. Has how these fields work changed in 3.x.x? I tried to look at the source and figure out what is happening exactly but I'm pretty hopeless on front end code.

Thanks everyone!