Not all many to many releations are getting saved

[Marvin-S]

Hello everybody,

I'm experiencing a strange behavior with the many to many relationship in my application.
I've worked through the Documentation and created 3 models:

1. OutsourcingPartner
2. BillingGroup
3. OutsourcingPartnerBillingGroup

Obviously the outsourcing partner and the billing group have a many to many relationship.

I recieve data from an API that returns the outsourcing partners with all their billing groups, but only some og the relations are getting saved.

My models are looking like this:

class OutsourcingPartner extends Model {
  static entity = 'outsourcingPartners';

  @Uid()
  declare id: string;

  @Str('')
  declare name: string;

  @BelongsToMany(
    () => BillingGroup,
    () => OutsourcingPartnerBillingGroup,
    'outsourcingPartner_id',
    'billingGroup_id'
  )
  declare billingGroups: BillingGroup[];
}

class BillingGroup extends Model {
  static entity = 'billingGroups';

  @Uid()
  declare id: string;

  @Str('')
  declare name: string;
}


class OutsourcingPartnerBillingGroup extends Model {
  static entity = 'outsourcingPartnerBillingGroups';

  static primaryKey = ['outsourcingPartner_id', 'billingGroup_id'];

  @Uid()
  declare outsourcingPartner_id: string;

  @Uid()
  declare billingGroup_id: string;
}

The data that I recieve from my API and pass to the save() function is the following:

[
    {
       "id":"36a11c85380b4afb994710ea63f99964",
       "name":"Luke Skywalker",
       "billingGroups":[
          {
             "id":"c8d8db2b1f4f4a069fa640c9c8cbe489",
             "name":"The Jedis",
             "__typename":"BillingGroup"
          }
       ],
       "__typename":"OutsourcingPartner"
    },
    {
       "id":"658b19b76a0f494590c21a7c27173d8e",
       "name":"Darth Vader",
       "billingGroups":[
          {
             "id":"95a87597ef124bbf9907f8f6074f8e94",
             "name":"Dark Forces",
             "__typename":"BillingGroup"
          }
       ],
       "__typename":"OutsourcingPartner"
    },
    {
       "id":"fabaecbb6ac045bb9b582244b889d309",
       "name":"Yoda",
       "billingGroups":[
          {
             "id":"4dedf55e5a8a4ce5affb1bc65b65de09",
             "name":"Jedi Council",
             "__typename":"BillingGroup"
          },
          {
             "id":"c8d8db2b1f4f4a069fa640c9c8cbe489",
             "name":"The Jedis",
             "__typename":"BillingGroup"
          }
       ],
       "__typename":"OutsourcingPartner"
    },
    {
       "id":"0284241ba3ed487eb8d12c0e86161fc0",
       "name":"Clone Trooper 0815",
       "billingGroups":[
          {
             "id":"95a87597ef124bbf9907f8f6074f8e94",
             "name":"Dark Forces",
             "__typename":"BillingGroup"
          }
       ],
       "__typename":"OutsourcingPartner"
    }
 ]

The result from the save() function is the following:

[
    {
        "id": "36a11c85380b4afb994710ea63f99964",
        "name": "Luke Skywalker",
        "billingGroups": [
            {
                "id": "c8d8db2b1f4f4a069fa640c9c8cbe489",
                "name": "The Jedis",
                "pivot_outsourcingPartnerBillingGroups": null
            }
        ]
    },
    {
        "id": "658b19b76a0f494590c21a7c27173d8e",
        "name": "Darth Vader",
        "billingGroups": [
            {
                "id": "95a87597ef124bbf9907f8f6074f8e94",
                "name": "Dark Forces",
                "pivot_outsourcingPartnerBillingGroups": null
            }
        ]
    },
    {
        "id": "fabaecbb6ac045bb9b582244b889d309",
        "name": "Yoda",
        "billingGroups": [
            {
                "id": "4dedf55e5a8a4ce5affb1bc65b65de09",
                "name": "Jedi Council",
                "pivot_outsourcingPartnerBillingGroups": {
                    "outsourcingPartner_id": "fabaecbb6ac045bb9b582244b889d309",
                    "billingGroup_id": "4dedf55e5a8a4ce5affb1bc65b65de09"
                }
            },
            {
                "id": "c8d8db2b1f4f4a069fa640c9c8cbe489",
                "name": "The Jedis",
                "pivot_outsourcingPartnerBillingGroups": null
            }
        ]
    },
    {
        "id": "0284241ba3ed487eb8d12c0e86161fc0",
        "name": "Clone Trooper 0815",
        "billingGroups": [
            {
                "id": "95a87597ef124bbf9907f8f6074f8e94",
                "name": "Dark Forces",
                "pivot_outsourcingPartnerBillingGroups": null
            }
        ]
    }
]

And the state store is looking like this after the call to the save() function:

{
  "outsourcingPartners": {
    "data": {
      "36a11c85380b4afb994710ea63f99964": {
        "id": "36a11c85380b4afb994710ea63f99964",
        "name": "Luke Skywalker"
      },
      "658b19b76a0f494590c21a7c27173d8e": {
        "id": "658b19b76a0f494590c21a7c27173d8e",
        "name": "Darth Vader"
      },
      "fabaecbb6ac045bb9b582244b889d309": {
        "id": "fabaecbb6ac045bb9b582244b889d309",
        "name": "Yoda"
      },
      "0284241ba3ed487eb8d12c0e86161fc0": {
        "id": "0284241ba3ed487eb8d12c0e86161fc0",
        "name": "Clone Trooper 0815"
      }
    }
  },
  "outsourcingPartnerBillingGroups": {
    "data": {
      "[\"36a11c85380b4afb994710ea63f99964\",\"c8d8db2b1f4f4a069fa640c9c8cbe489\"]": {
        "outsourcingPartner_id": "36a11c85380b4afb994710ea63f99964",
        "billingGroup_id": "c8d8db2b1f4f4a069fa640c9c8cbe489"
      },
      "[\"658b19b76a0f494590c21a7c27173d8e\",\"95a87597ef124bbf9907f8f6074f8e94\"]": {
        "outsourcingPartner_id": "658b19b76a0f494590c21a7c27173d8e",
        "billingGroup_id": "95a87597ef124bbf9907f8f6074f8e94"
      },
      "[\"fabaecbb6ac045bb9b582244b889d309\",\"4dedf55e5a8a4ce5affb1bc65b65de09\"]": {
        "outsourcingPartner_id": "fabaecbb6ac045bb9b582244b889d309",
        "billingGroup_id": "4dedf55e5a8a4ce5affb1bc65b65de09"
      }
    }
  },
  "billingGroups": {
    "data": {
      "c8d8db2b1f4f4a069fa640c9c8cbe489": {
        "id": "c8d8db2b1f4f4a069fa640c9c8cbe489",
        "name": "The Jedis"
      },
      "95a87597ef124bbf9907f8f6074f8e94": {
        "id": "95a87597ef124bbf9907f8f6074f8e94",
        "name": "Dark Forces"
      },
      "4dedf55e5a8a4ce5affb1bc65b65de09": {
        "id": "4dedf55e5a8a4ce5affb1bc65b65de09",
        "name": "Jedi Council"
      }
    }
  }
}

So as you can see, only three relations are getting stored, but I have five in total. And also if I query the model from the repo, I only recieve three relations.

I'm pretty sure there is some kind of misconfiguration from my side, but I don't know where.

Any help is really appreciated.

[CodeDredd]

Hey @Marvin-S ,

i tried your code and got the right result:

{
  outsourcingPartnerBillingGroups: {
    data: {
      '["36a11c85380b4afb994710ea63f99964","c8d8db2b1f4f4a069fa640c9c8cbe489"]': [Object],
      '["658b19b76a0f494590c21a7c27173d8e","95a87597ef124bbf9907f8f6074f8e94"]': [Object],
      '["fabaecbb6ac045bb9b582244b889d309","4dedf55e5a8a4ce5affb1bc65b65de09"]': [Object],
      '["fabaecbb6ac045bb9b582244b889d309","c8d8db2b1f4f4a069fa640c9c8cbe489"]': [Object],
      '["0284241ba3ed487eb8d12c0e86161fc0","95a87597ef124bbf9907f8f6074f8e94"]': [Object]
    }
  },
  billingGroups: {
    data: {
      c8d8db2b1f4f4a069fa640c9c8cbe489: [Object],
      '95a87597ef124bbf9907f8f6074f8e94': [Object],
      '4dedf55e5a8a4ce5affb1bc65b65de09': [Object]
    }
  },
  outsourcingPartners: {
    data: {
      '36a11c85380b4afb994710ea63f99964': [Object],
      '658b19b76a0f494590c21a7c27173d8e': [Object],
      fabaecbb6ac045bb9b582244b889d309: [Object],
      '0284241ba3ed487eb8d12c0e86161fc0': [Object]
    }
  }
}

If you can give me a working reproduction then i can help you. 🙏 Right now it seems some important information is missing.

[Marvin-S]

I finally figured out what is causing this issues 🎉
The problem is that Apollo transforms the data that it receives from the API to have single instances for the data. So you only have one instance of Billing Group 1, one instance of Billing Group 2 and so on. So Apollo realizes, that the Billing Group that is assigned to multiple Outsourcing Partners is the same and only creates one instance for it.

You can reproduce this behavior with the following data:

const billingGroups = [
      {
        id: 1,
        name: 'The Jedis',
        __typename: 'BillingGroup',
      },
      {
        id: 2,
        name: 'Jedi Council',
        __typename: 'BillingGroup',
      },
      {
        id: 3,
        name: 'Dark Forces',
        __typename: 'BillingGroup',
      },
    ]

    const outsourcingPartners = [
      {
        id: 1,
        name: 'Luke Skywalker',
        billingGroups: [billingGroups[0], billingGroups[1], billingGroups[2]],
        __typename: 'OutsourcingPartner',
      },
      {
        id: 2,
        name: 'Darth Vader',
        billingGroups: [billingGroups[0], billingGroups[1], billingGroups[2]],
        __typename: 'OutsourcingPartner',
      },
      {
        id: 3,
        name: 'Yoda',
        bbillingGroups: [billingGroups[0], billingGroups[1], billingGroups[2]],
        __typename: 'OutsourcingPartner',
      },
      {
        id: 4,
        name: 'Clone Trooper 0815',
        billingGroups: [billingGroups[0], billingGroups[1], billingGroups[2]],
        __typename: 'OutsourcingPartner',
      },
    ]

If you save the data like this, you can reproduce the problem:

const repo = useRepo(OutsourcingPartner)
    repo.save(outsourcingPartners)

I've updated my PR, to remove dependencies from Apollo and GraphQL in general, so we can focus on the root of the problem :)

[CodeDredd]

Awesome work Marvin! Now i am going to look into a solution for this.

[CodeDredd]

Ok....i don't know if there is really an error in the normalizr. But i found a good solution.
Instead of using loadash _clonedeep use https://github.com/rhysd/fast-json-clone or if you have Date objects https://github.com/davidmarkclements/rfdc. With that finally the refernces are removed.
Since we need a extensible object anyway, it seems to be the correct way.

The question maybe that i still have is if is should include this into the save function to check if the object is extensible and use this package.

[Marvin-S]

Yeah, I think you're right. I would say Normalizr works as designed in this case.

I will take a look at the rfdc solution, because we might have Date objects in the future.

I also thought about where this 'dereferencing' (if this is even a word 😄) should be placed: in Apollo, in the app code or inside pin-orm. I think one could argue for every option, but I think those references inside the object are not that uncommon, so I personally think that it would be a good place to implement this 'dereferencing' inside pinia-orm.

[CodeDredd]

Yeah i think i need an option for this. I am just thinking about if this is going to be an only graphql problem. Because in REST you will never have this problem.