Skip to content

Limit the number of results pages which can be returned, to try to deter bots #164

@m-i-l

Description

@m-i-l

Bots regularly try to get crazy amounts of results simply by appending the page parameter, without actually checking if there are that many pages of results, wasting a lot of CPU resources in the process. For example, a bot has spent hours searching through 10s of 1000s of pages of results for the query "argos garden furniture habitat", via e.g. GET /search/?q=argos+garden+furniture+habitat&page=28802, despite there not being one single result for that query.

Really should return a 404 if the page parameter is above a certain value, e.g. 100, or at the very least not waste CPU time actually performing the search.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions