Provide a way to avoid circular import of serializers

[mrodal]

Its a common case where you have modules where both refer to each other.

For example, modules users and posts. On users you have User and on posts you have Post and Attachment

# users
class UserSerializer:
    posts = PostSerializer()
    profile_attachments = AttachmentSerializer()


#posts
class PostSerializer:
    user = UserSerializer()
    attachments = AttachmentSerializer()

class AttachmentSerializer:
    name = CharField()
    size = IntegerField()

In Django there's a way to declare Foreign Keys that avoids circular dependencies, declaring the referenced model by a string:

profile_attachment = models.ForeignKey('posts.Attachment', on_delete=models.CASCADE)

Could DRF offer something like that for serializers? Something like

profile_attachment = ModelSerializer(serializer='posts.AttachmentSerializer')

[jvhellemondt]

Hi mrodal!

I managed to achieve the same by implementing the following in one of the "circular" serializers: DRF: Reverse relations
Django: Following relationships “backward”

Though I do think it's quite complex.

[C0DK]

So i actually have the same case in a project i work on, but you really shouldn't. best practice, from a software engineering and architecture course is to have a one way dependency. If possible atleast. If there is a very close coupling between the two apps they should maybe be one app, alternatively they should have a common parent. You could also move your serializer into a sub-app if applicable.

I might be wrong, but AFAIK that is best practice.

[mrodal]

I agree, but sometimes you also might want to split a large file in a given app into multiple smaller files, grouping serializers into rough groups, each group having serializers related to some particular funcionality, but that is not totally uncoupled from the rest of the groups.. for example a group containing serializers related to users like UserSerializer, AdminSerializer, RoleSerializer.. another group related to posts like PostSerializer, AttachmentSerializer, CommentSerializer.. and another group related to sales, like SaleListingSerializer, SaleSerializer, and ListingSnapshotSerializer..

Those are 3 groups that have serializers tightly coupled within each group, but they could also have cross group references, for example UserSerializer could include a list of the user Posts and Purchases, and PostSerializer and SaleSerializer could have a reference to the User that created/purchased it.

In this example there are just 9 serializers, they could be in the same file to work around the problem, but with larger apps with a lot more of serializers it might start to be desirable to split it in some way. I feel its wrong that this problem forces the layout of the app to be this way, and the developer cant split it the way he feels its right.

[xordoquy]

I'm surprised this does not end up in infinite loop.

[tomchristie]

First step to addressing this would be for someone to demo this as a code-snippet or third party package.

I suppose that might look something like this?...

class LazyRefSerializer(serialisers.BaseSerializer):
    def __init__(self, ref):
        self._reference_as_string = ref
        self._reference_as_serializer = None

    def to_representation(self):
        if self._reference_as_serializer is None:
            # Lazily create a model serializer
            model_ref = import_from_string(self._reference_as_string)
            class DynamicSerializer(ref):
                class Meta:
                    model = model_ref
            self. _reference_as_serializer = DynamicSerializer

        return self._reference_as_serializer.to_representation()

If anyone wants to expand on that and test it, then we could link to a code snippet or third party package from the docs.

[mrodal]

Thanks for giving this a look!
If I understand correctly, following the example from the original post, it would be used like this, right?

# users
class UserSerializer:
    posts = LazyRefSerializer('models.Post')
    profile_attachments = LazyRefSerializer('models.Attachment')


# posts
class PostSerializer:
    user = LazyRefSerializer('models.User')
    attachments = AttachmentSerializer()

class AttachmentSerializer:
    name = CharField()
    size = IntegerField()

So the reference passed to the LazyRefSerializer points to the original model, right? How would it work for more customized serializers? because DynamicSerializer only sets the model but not for example read_only_fields and such

This could work, but originally I was thinking of a way to reference a serializer instead of a model, that way you can customize it all you want, and just use it but without importing it

Based on your example, I think this could work that way, what do you think?

class LazyRefSerializer(serializers.ModelSerializer):
    def __init__(self, ref, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs
        self._reference_as_string = ref
        self._reference_as_serializer = None

    def __getattr__(self, item):
        return getattr(self._reference_as_serializer, item)

    def __getattribute__(self, attr):
        # When first trying to use its attributes, it imports and initializes the original serializer
        # The 'not in' check is to avoid infinite loops. _creation_counter is called when initializing the serializer which uses this LazyRefSerializer field
        if attr not in ['args', 'kwargs', '_reference_as_string', '_reference_as_serializer', '_creation_counter'] and self._reference_as_serializer is None:
            referenced_serializer = import_from_string(self._reference_as_string, '')
            self._reference_as_serializer = referenced_serializer(*self.args, **self.kwargs)
            self.__class__ = referenced_serializer
            self.__dict__.update(self._reference_as_serializer.__dict__)
        return object.__getattribute__(self, attr)

I tested it a little bit and worked, but it surely needs further testing from more use cases

It would be used like this:

# users
class UserSerializer:
    posts = LazyRefSerializer('serializers.PostSerializer')
    profile_attachments = LazyRefSerializer('serializers.AttachmentSerializer')


# posts
class PostSerializer:
    user = LazyRefSerializer('serializers.UserSerializer')
    attachments = AttachmentSerializer()

class AttachmentSerializer:
    name = CharField()
    size = IntegerField()

[sharky98]

Hello,

I am stuck in a similar situation where my "read" serializer are giving circular import issues. I wanted to try this code, but I can't figure out the where the "import_from_string()" come from (python 3.9.4). Any idea which module or custom function it is from?

Regards!

[tuenut]

try this or this

[Dooweed]

Hello,

I am stuck in a similar situation where my "read" serializer are giving circular import issues. I wanted to try this code, but I can't figure out the where the "import_from_string()" come from (python 3.9.4). Any idea which module or custom function it is from?

Regards!

It is in rest_framework.settings, (djangorestframework==3.13.1)