Replace/switch/enhance the way keywords for a page are set.

[jgadelange]

Is your feature request related to a problem? Please describe.
Currently the meta keywords field is used to discover the keywords for a specific page. Since this field isn't widely used by search engines anymore we don't have it on our websites, so it is currently impossible for us to set the keywords for the checks, which in turn leads to a lot of failing checks.

Describe the solution you'd like
I would like to be able to for each page fill in the keywords on the check seo page, that then will be used in the checks. For this I would like to add a Keywords model with an URL or path and a keywords field. When on the check seo page you can then click on the shown keywords to edit them.

Probably we also want to be able to configure this behaviour to also allow for the current behaviour. I think this could be achieved by a setting (e.g. DJANGO_CHECK_SEO_KEYWORDS_DISCOVERY_METHOD) with support by default for the options:

• meta keywords
• from db

Some stuff I'm not sure about:

1. Should the models live in a separate app (I think it should be part of this package), so database tables aren't created when someone doesn't want to use the database feature?
2. Should we also add support for having another keywords retrieval method alltogether? So people can write their own python method and link that in the setting?
3. Should we be able to have a fallback method? So first check in db, if not there, check in meta field. Or should we combine those? Should having this fallback be the default behaviour?
4. In my suggested solution you would be able to click on the keywords to edit them. I think this would be an interesting feature independent on which keywords discovery method you use. How can we create this in such way that it is versatile?
5. A big downside to db solution where you store the path to link the keywords is that if a page is moved the keywords don't move with it (and you will have "ghost keywords" for pages that no longer exist). Do we think this is a problem?

Describe alternatives you've considered
I couldn't think of other feasible solutions.

Additional context
N/A

[jgadelange]

For the 5th bullet it might be possible to have a generic relation to the model that is used on the page and link the keywords based on that. However that would mean we have a way to determine the most relevant model used, I'm not sure how doable that would be.

I think create the path<->keyword table first and later possible extend/replace that would be the best course of action. Unless someone has some genius ideas how to incorporate something else immediately.

[benzkji]

Second this. And would add another option for discovery: from title tag. This would not work if a page is not optimized at all, but if it is, it would. And it would just work as is, without additional models or other things. One would just need to define some "extraction" logic, like "separate by comma and /", or maybe a customizable callable...?

[corentinbettiol]

Hi, thank you @jgadelange for your issue and @benzkji for the comment (and happy new year)!

I think that's a great idea, we should add this behavior in the app :)

1. Should the models live in a separate app (I think it should be part of this package), so database tables aren't created when someone doesn't want to use the database feature?

I don't know if it's the case for you, but our websites are riddled with empty tables in their db, so I think it's not an issue to add an empty table in the app.

2. Should we also add support for having another keywords retrieval method alltogether? So people can write their own python method and link that in the setting?

We can interpret the setting as an import path (kind of dirty but hey the plugin system of django check seo already does this), so every user can configure it like he or she wants. We could document the 2 "official" ways to do this, eg. by putting django_check_seo.utils.keywords_discovery.meta_keywords & django_check_seo.utils.keywords_discovery.model_keywords in the README.

3. Should we be able to have a fallback method? So first check in db, if not there, check in meta field. Or should we combine those? Should having this fallback be the default behaviour?

We should definitely add the DJANGO_CHECK_SEO_KEYWORDS_DISCOVERY_METHOD with a default to django_check_seo.utils.keywords_discovery.meta_keywords in order to not break existing setups, and add something like django_check_seo.utils.keywords_discovery.model_keywords to use the Models (and write it on the README).

4. In my suggested solution you would be able to click on the keywords to edit them. I think this would be an interesting feature independent on which keywords discovery method you use. How can we create this in such way that it is versatile?

We could use something similar to the autocomplete_fields in Django admin, and a refresh button next to the input.

5. A big downside to db solution where you store the path to link the keywords is that if a page is moved the keywords don't move with it (and you will have "ghost keywords" for pages that no longer exist). Do we think this is a problem?

I think it's safe to assume that even if we have a few hundreds dead objects stored in the db it won't kill it, so it may be a problem for laterTM. What we can do is add a management command that tries to resolve each url using the Django test system, and to remove the object if the page is not found.

We can add 2 models: Page & Keyword. A page only store a path & a list of Keywords (m2m) (and a language ?), and a Keyword is a simple char field. That way we can search for the Page, then search for the related Keywords (using prefetch_related to improve db access, even if this is not a big app).

The idea behind the "2 models" concept is that once a Keyword was created one time, it could be reused (and it even let the user save a few keystrokes by suggesting the keyword once he or she start typing it on the auto-completed field :P).

[jgadelange]

Sounds good!

-1. I can be a bit puristic in this kind of stuff ;P. I agree that having some unused db tables isn't that big of an issue ;)

-5. The bigger problem of the two for me would be that the keywords are lost when a page is moved. I agree that having "ghost keywords" is not really a problem. But having to re-enter the keywords for a page you just moved could be really frustrating. So for that it could be desireable to also connect the (SEO)Page model trough a generic FK to the model the pages displays (so for django-cms pages that would be the Page model or an Event model for for a calendar application)