Recursive tagged unions fail to create serializer

[DenSinH]

Hello again,

We have stumbled upon another interesting edge case: we are trying to load a model which has a self-referential tagged union. I made a small example that fails:

from typing import Literal, Annotated

from pydantic_xml import BaseXmlModel, element, attr
from pydantic import Field


class Other(BaseXmlModel, tag="other"):
    name: Literal["other"] = attr(default="other")


class Test(BaseXmlModel, tag="test"):
    name: Literal["hi"] = attr(default="hi")
    nested: list[Annotated["Test | Other", Field(discriminator="name")]] = element(default=None)


text = """
    <test name="hi">
    <nested name="hi">
    </nested>
    <nested name="hi">
    </nested>
    <nested name="hi">
    </nested>
    <nested name="hi">
    </nested>
    </test>
"""
print(Test.from_xml(text))

The expecte behavior is that this just loads™. Sadly, it crashes trying to construct the serializer for Test:

Traceback (most recent call last):
  File "C:\projects\playground\mdsectioning.py", line 11, in <module>
    class Test(BaseXmlModel, tag="test"):
  File "C:\projects\playground\pydantic-xml\pydantic_xml\model.py", line 387, in __new__
    cls.__build_serializer__()
  File "C:\projects\playground\pydantic-xml\pydantic_xml\model.py", line 494, in __build_serializer__
    serializer = Serializer.parse_core_schema(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\serializer.py", line 184, in parse_core_schema
    return cls.select_serializer(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\serializer.py", line 241, in select_serializer
    return factories.model.from_core_schema(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\factories\model.py", line 453, in from_core_schema
    return ModelSerializer.from_core_schema(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\factories\model.py", line 91, in from_core_schema
    fields_serializers[field_name] = Serializer.parse_core_schema(model_field['schema'], field_ctx)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\serializer.py", line 184, in parse_core_schema
    return cls.select_serializer(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\serializer.py", line 245, in select_serializer
    return factories.homogeneous.from_core_schema(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\factories\homogeneous.py", line 133, in from_core_schema
    return ElementSerializer.from_core_schema(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\factories\homogeneous.py", line 31, in from_core_schema
    inner_serializer = Serializer.parse_core_schema(items_schema, ctx)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\serializer.py", line 184, in parse_core_schema
    return cls.select_serializer(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\serializer.py", line 265, in select_serializer
    return factories.tagged_union.from_core_schema(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\factories\tagged_union.py", line 126, in from_core_schema
    return ModelSerializer.from_core_schema(schema, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\projects\playground\pydantic-xml\pydantic_xml\serializers\factories\tagged_union.py", line 37, in from_core_schema
    assert isinstance(model_serializer, factories.model.ModelSerializer), "unexpected model serializer type"
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: unexpected model serializer type

From what I could tell, it does the following: it tries to add serializers for all possible options in the union, gets to the model itself and then gets None, as it is not created yet. I am not sure how this is solved for directly recursive models, but it does not work in this case.

[dapper91]

@DenSinH Hi,

That is the known problem with self-referenced tagged union. That happens because the tagged union must know the discriminating field, which is in your case a part of Test model that is under construction yet.

That can be solved by separating root model and union types:

from typing import Literal, Annotated

from pydantic_xml import BaseXmlModel, element, attr
from pydantic import Field


class Other(BaseXmlModel, tag="nested"):
    name: Literal["other"] = attr(default="other")


class Hi(BaseXmlModel, tag="nested"):
    name: Literal["hi"] = attr(default="hi")


class Test(Hi, tag="test"):
    nested: list[Annotated["Hi | Other", Field(discriminator="name")]] = element(default=None)


text = """
    <test name="hi">
    <nested name="hi">
    </nested>
    <nested name="hi">
    </nested>
    <nested name="hi">
    </nested>
    <nested name="hi">
    </nested>
    </test>
"""
print(Test.from_xml(text))

[DenSinH]

Ah, I didn't know this was a known issue. I figured this was the problem, I thought perhaps it could be solved in a similar way to more direct self-referential models. The way I solved it now was to create a custom xml serializer and validator, where I create a local model with just the full tagged union and then re-use that loader. It's because I really need the model to be self referential. It works but isn't super clean.