Automatically detect models null=True, blank=True, unique=True and set serializers allow_blank=False

[kiraware]

TL;DR
If models blank=True, null=True, and unique=True then set serializers allow_blank=False

The default serializers.ModelSerializer class provide very close to models field definition. This feature is very nice, but it came into problem if we set models field with blank=True, null=True, unique=True and the serializer will set allow_blank=True, allow_null=True, required=False. The problem with this is when we create or update with blank, the serializer will save it with empty string "", for the first time it's not a problem to the database since no one field in database contain empty string "", but if we create or update another field with blank again, the database will raise IntegrityError due to empty string "" which is originally caused by unique=True.

Suppose we have models

class Foo(models.Model):
    field0 = models.CharField(max_length=32, blank=False, null=False, unique=False)
    field1 = models.CharField(max_length=32, blank=False, null=False, unique=True)
    field2 = models.CharField(max_length=32, blank=False, null=True, unique=False)
    field3 = models.CharField(max_length=32, blank=False, null=True, unique=True)
    field4 = models.CharField(max_length=32, blank=True, null=False, unique=False)
    # unlikely to be happen
    field5 = models.CharField(max_length=32, blank=True, null=False, unique=True)
    field6 = models.CharField(max_length=32, blank=True, null=True, unique=False)
    field7 = models.CharField(max_length=32, blank=True, null=True, unique=True)

And serializers

class FooSerializer(serializers.ModelSerializer):
    class Meta:
        model = Foo
        fields = ['field0', 'field1', 'field2', 'field3', 'field4', 'field5', 'field6', 'field7', ]

We have this field set

>>> FooSerializer()
FooSerializer():
    field0 = CharField(max_length=32)
    field1 = CharField(max_length=32, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
    field2 = CharField(allow_null=True, max_length=32, required=False)
    field3 = CharField(allow_null=True, max_length=32, required=False, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
    field4 = CharField(allow_blank=True, max_length=32, required=False)
    field5 = CharField(allow_blank=True, max_length=32, required=False, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
    field6 = CharField(allow_blank=True, allow_null=True, max_length=32, required=False)
    field7 = CharField(allow_blank=True, allow_null=True, max_length=32, required=False, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
>>>

Here is the summary of model and serializer default behaviour with logical table

+----------------------------------------------------------------------------------------------------------+
|          |             models            |                         serializers                           |
|  field   |-------------------------------|---------------------------------------------------------------|
|          |  blank  |  null  |   unique   |  allow_blank  |  allow_null  | required  |   UniqueValidator  |
|----------|-------------------------------|---------------------------------------------------------------|
|  field0  |    F    |    F   |      F     |       F       |      F       |     T     |         -          |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field1  |    F    |    F   |      T     |       F       |      F       |     T     |        SET         |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field2  |    F    |    T   |      F     |       F       |      T       |     F     |         -          |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field3  |    F    |    T   |      T     |       F       |      T       |     F     |        SET         |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field4  |    T    |    F   |      F     |       T       |      F       |     F     |         -          |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field5  |    T    |    F   |      T     |       T       |      F       |     F     |        SET         |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field6  |    T    |    T   |      F     |       T       |      T       |     F     |         -          |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field7  |    T    |    T   |      T     |       T       |      T       |     F     |        SET         |
+----------------------------------------------------------------------------------------------------------+

The problem is in field5 and field7, due to their unique is set to True, saving the blank or empty string "" into database for the first time is not a problem, but not for second time. The database will raise IntegrityError due to UNIQUE constraint failed by empty string "".

The case with field5 is unlikely to be happen in real world, and if this really happen, we couldn't fix it by set allow_blank to False and allow_null to True. Due to models definition set the null to False, hence we couldn't fix it

+----------------------------------------------------------------------------------------------------------+
|          |             models            |                         serializers                           |
|  field   |-------------------------------|---------------------------------------------------------------|
|          |  blank  |  null  |   unique   |  allow_blank  |  allow_null  | required  |   UniqueValidator  |
|----------|-------------------------------|---------------------------------------------------------------|
|  field5  |    T    |    F   |      T     |       T       |      F       |     F     |        SET         |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field5  |    T    |    F   |      T     |       F       |      T       |     F     |        SET         | # IMPOSSIBLE FIX
+----------------------------------------------------------------------------------------------------------+

The case with field7 is likely to be happen, and if this really happen we can fix it by set allow_blank to False. Therefore the serializers won't accept blank or empty string "", so that the database raising IntegrityError is unlikely to be happen

+----------------------------------------------------------------------------------------------------------+
|          |             models            |                         serializers                           |
|  field   |-------------------------------|---------------------------------------------------------------|
|          |  blank  |  null  |   unique   |  allow_blank  |  allow_null  | required  |   UniqueValidator  |
|----------|-------------------------------|---------------------------------------------------------------|
|  field7  |    T    |    T   |      T     |       T       |      T       |     F     |        SET         |
|----------|---------+--------+------------|---------------+--------------+-----------+--------------------|
|  field7  |    T    |    T   |      T     |       F       |      T       |     F     |        SET         | # POSSIBLE FIX
+----------------------------------------------------------------------------------------------------------+

After all, the developer could fix the field7 case by explicitly set allow_blank=False, allow_null=True, required=False either in extra_kwargs Meta field or in serializer definition. But i think it would be nice if DRF automatically detect case in field7 and set the allow_blank to False so that the developer don't have to set it manually

Thanks for your attention!

[auvipy]

can you help us to dig it further and figure out some possible approach for this?

[kiraware]

I have two approach which might works as expected but needed more test for stable. We can insert this code into rest_framework.serializers.ModelSerializer on get_fields() method at this line https://github.com/encode/django-rest-framework/blob/master/rest_framework/serializers.py#L1081 after got exact field mapping from models and before extra_kwargs replace it. The reason we should place this code at this line because we want the default field mapping from models and then detect blank=True, null=True, unique=True. If these all set to True then set allow_blank to False. And then still allow developer to override it through extra_kwargs

First approach

...
# Detect if allow_blank=True and allow_null=True
if field_kwargs.get('allow_blank') == True and field_kwargs.get('allow_null') == True:
    # Find instance of UniqueValidator
    from rest_framework.validators import UniqueValidator
    for validator in field_kwargs.get('validators', []):
        # If there is UniqueValidator instance, then set allow_blank to False
        if isinstance(validator, UniqueValidator):
            field_kwargs.update({'allow_blank': False})
            break
...

Or with second approach

...
field_info = info.fields[field_name]
# Detect if blank=True, null=True, unique=True then set allow_null to False
if field_info.blank and field_info.null and field_info.unique:
    field_kwargs.update({'allow_blank': False})
...

Then we got serializers

>>> FooSerializer()
FooSerializer():
    field0 = CharField(max_length=32)
    field1 = CharField(max_length=32, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
    field2 = CharField(allow_null=True, max_length=32, required=False)
    field3 = CharField(allow_null=True, max_length=32, required=False, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
    field4 = CharField(allow_blank=True, max_length=32, required=False)
    field5 = CharField(allow_blank=True, max_length=32, required=False, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
    field6 = CharField(allow_blank=True, allow_null=True, max_length=32, required=False)
    field7 = CharField(allow_blank=False, allow_null=True, max_length=32, required=False, validators=[<UniqueValidator(queryset=Foo.objects.all())>])
>>>

Anyway i am not really sure if this happen ONLY for UniqueValidator instance, maybe for another Unique*Validator example UniqueTogetherValidator, UniqueForDateValidator etc.