ensure_mongo_indexes issue with mongo 8

[ehiggs]

There is new behaviour when creating indexes in mongo 8 that causes a bad interaction with ensure_mongo_indexes. If an index is hidden, then the create index returns an IndexOptionsConflict error causing Eve to mistakenly think it needs to recreate the index.

The following code works in mongo 7 but fails in mongo 8:

db["mycollection"].createIndex({field1:1, field2:1}, {name: "myindex"})
myindex
db["mycollection"].hideIndex("myindex")
{ hidden_old: false, hidden_new: true, ok: 1 }
db["mycollection"].createIndex({field1:1, field2:1}, {name: "myindex"})
MongoServerError[IndexOptionsConflict]: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { field1: 1, field2: 1 }, nam...

This means that the following code fails in _create_index:

   for coll in colls:
        try:
            coll.create_index(list_of_keys, **kw)
        except pymongo.errors.OperationFailure as e: # <------ this is hit because we get an IndexOptionsConflict
            if e.code in (85, 86):
                # raised when the definition of the index has been changed.
                # (https://github.com/mongodb/mongo/blob/master/src/mongo/base/error_codes.err#L87)

                # by default, drop the old index with old configuration and
                # create the index again with the new configuration.
                coll.drop_index(name)
                coll.create_index(list_of_keys, **kw)
            else:
                raise

Workaround(s):

1. Don't hide indexes if one is using the mongo_index functionality of eve.
2. If something needs to be hidden, comment the index out of the eve mongo_indexes section of the domain schema.

[nicolaiarocci]

thanks for the heads up. Is there we can do code-wise to circumvent this problem? Would you be willing to open a pull request? I'll be happy to review it.

[y3h7rv]

Hi everyone,

I've thoroughly investigated this issue and have submitted a Pull Request with a proposed fix.

The solution addresses the IndexOptionsConflict on MongoDB 8+ by checking if the index is merely hidden. If so, it un-hides the index via a collMod command instead of performing a risky drop/recreate operation. I have also included a regression test to cover this specific scenario.

The PR is available for review here: #1556

Looking forward to your feedback. Thanks!

[nicolaiarocci]

I think the workarounds suggested by @ehiggs are perfectly fine in this case. As I understand them, hidden indexes use case is that of a temporary experiment anyways.