Migrating custom search worker from 8.5.11 to 9.5.33

[threadedstream]

Hi! When migrating mkdocs-material to version 9.5.33 we realized that custom search worker integration process has changed. In older version, we used the following construction to do that

{% extends "base.html" %}

{% block config %}
  {{ super() }}
  <script>
    var __search = {
      worker: "/assets/js/generated/search/worker.js"
    }
  </script>
{% endblock %}

Alas, it doesn't work that way in newer versions. So, how do I solve this issue?

[kamilkrzyskow]

Hi @threadedstream,
I didn't even know that there was a support for a custom worker in the past 🤔
Looking at the source code, the idea has been mostly abandoned. [EDIT: So custom hooks, aren't supported anymore like I assumed, but the rest of the answer is still valid in saying you can overwrite the current worker with your own]

There is a dangling dead URL here:

mkdocs-material/src/templates/assets/javascripts/integrations/search/worker/_/index.ts

Line 63 in b2d235e

* @see https://bit.ly/3igvtQv - How to implement custom search

And the search URL is taken from the config object, which is passed in the base.html template:

mkdocs-material/src/templates/assets/javascripts/components/search/_/index.ts

Line 110 in b2d235e

const worker$ = setupSearchWorker(config.search, index$)

mkdocs-material/src/templates/base.html

Line 420 in b2d235e

"search": "assets/javascripts/workers/search.js" | url,

I don't know what's the purpose of the worker, nor what's the API, what data needs to be passed over etc.
But the code should be somewhere around here:
https://github.com/squidfunk/mkdocs-material/tree/master/src/templates/assets/javascripts/integrations/search

You don't have to write it in TypeScript, but perhaps it would be easier, I'm not sure TBH.
To replace the worker set in the config, you can use a hook to write a new one under the same path as it's in the base.html

Or maybe I'm missing something.
@squidfunk Does the current search implementation support custom search workers / hooks, or is only one worker hardcoded and the users have to replace it? What's the compatibility of the search worker API compared to the previous implementation?

[squidfunk]

Search worker support was removed in v9. More context in #4980. The worker approach had severe limitations, and the architecture did not fix the necessary changes we needed to make for v9. Search customization will be tackled as part of #6307.

[threadedstream]

Oh, I see it now. Well, does it mean we should abandon the idea of overriding search worker altogether?

[squidfunk]

Currently, yes, or you use an older version.

[threadedstream]

Okay, thank you. We'll be tracking the progress of #6307 then.

[threadedstream]

@squidfunk I didn't notice it's closed. Am I right that this search engine enhancement already present in latest version of mkdocs-material?

[kamilkrzyskow]

No, it's not.
It's part of the backlog.

• Backlog: see open change requests #8200
• Backlog

[threadedstream]

@kamilkrzyskow got it, thank you very much!

[threadedstream]

@squidfunk @kamilkrzyskow I'm sorry for reopening the discussion once again. I wonder if it's possible to develop my own search plugin, that would coexist with mkdocs-material and use all its perks like styles, themes, icons, etc, but at the same time have the ability to overwrite the worker behaviour. We really need to upgrade to newer mkdocs-material because version 8.5.11 has vulnerabilities and security guys are pushing us to perform upgrade. I actually tried to develop my own plugin by modifying search.html a little bit and embedding it via header.html, basically I tried to put search functionality of 8.5.11 into 9.5.33. Unfortunately, it didn't work, because material overwrites everything and renders its own htmls.

[kamilkrzyskow]

I wonder if it's possible to develop my own search plugin

Sure. Everything here can be replaced, but depending on the severity of the changes you either have to develop a full blown theme customization or use hacks. The "search" part on the website is managed by the JavaScript. The Python plugin generates the search_index.json file from MkDocs documentation pages, then the JSON files is loaded in the frontend JS.

There are no guides for this. As I mentioned in the marked answer, you can replace the assets/javascripts/workers/search.js file, but the analysis of the what, why, how, of the file "API" inside the file to make it "compliant" with the rest of the current theme is left up to you.

I believe there was some kind of discussion in the main MkDocs repository, where somebody advertised some sort of AI search tool, and advertised their guide for integrating such a thing. This completely bypasses the search here.

We really need to upgrade to newer mkdocs-material because version 8.5.11 has vulnerabilities and security guys are pushing us to perform upgrade

You could fork the theme at version 8.5.11 and just update the security dependencies. In case of deprecation, there should be an alternative to handle the same logic in the updated tools.

Whether this approach is easier than replacing the search is unknown to me, but feels easier, as fixing existing logic to work with newer tooling should be quicker than implementing completely new logic.

Unfortunately, it didn't work, because material overwrites everything and renders its own htmls.

This should be fully supported without hacks. Hard to say what is wrong without a reproduction. MkDocs and Jinja templating is straightforward. There is a chain of paths with templates, mkdocs, mkdocs-material, your_overrides. Then a template.html file accesses a path like partials/actions.html and looks for it in reverse order, so if that path is in your_overrides then it will be loaded.

https://squidfunk.github.io/mkdocs-material/customization/#extending-the-theme

---

In the other discussion it seemed that a simple "dummy" to hide the warning was enough

• Deprecation blocks strict build when migrating to newer version #8310

But now it looks like you have clear hard requirements of what the search needs to provide. So if you require help, lay out the whole requirements with example data, and someone might be willing to help, but no promises ✌️