Algolia cannot index notebooks due to redirect base url

[iishiishii]

query to start url: https://www.neurodesk.org/example-notebooks/ will return <meta http-equiv="Refresh" content="0; url=intro.html" />;

[iishiishii]

In algolia config, edited startUrls to include https://www.neurodesk.org/example-notebooks/intro.html to avoid redirect, as well as ignoreCanonicalTo: true.

Included this element for crawler to extract record from example notebook page

            content: [
              ...
              "article section",
            ],

This helps to crawl the intro page but notebooks pages are too large.

[stebo85]

do we know where this is stuck?

[stebo85]

instead of fixing large notebook indexing, what we do is:

1. automatically create a page in the neurodesk.org website that lists all example notebooks with a link on a normal page, that then agolia can search
2. remove indexing of agolia of notebooks