Starting a series where I would be compiling some of the bugs I run into during my software development projects. These bugs have usually given me a hard time since there seems to be nothing wrong at face value, but on deeper analysis, they expose some conceptual mistake on my part or some way the system works which was not known to me.

Overview

As part of a bigger project involving the management and billing of real estate projects, I was building a system for user management. I often use Django+DRF or FastAPI if I am using python for the backend task, but this time I thought of trying Django Ninja, a tool for building APIs with Django and Python 3.6+ type hints, along with Pydantic and async support, and automatic OpenAPI docs generation. So in a way, it brings the good parts of FastAPI to django ecosystem.

For the endpoint where a superuser can create a new user, the flow was simple:

  1. Accept a CreateUserSchema in the payload. 
     class CreateUserSchema(Schema):
     email: EmailStr
     company_name: str
     name_of_person_in_charge: str
     person_in_charge_title: str
     contact: PhoneNumber = None
     avatar: HttpUrl = None
     address: str = None
  2. Check if the user with the given email does not already exist.
  3. Create a user object using User.objects.create(), and assign it a random password. 
     class User(AbstractBaseUser, PermissionsMixin):
     id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
     company_name = models.CharField(max_length=250)
     name_of_person_in_charge = models.CharField(max_length=250)
     person_in_charge_title = models.CharField(max_length=250)
     contact = PhoneNumberField(null=True)
     email = models.EmailField(max_length=200, unique=True)
     address = models.CharField(null=True, max_length=250)
     avatar = models.URLField(null=True)
     created_at = models.DateTimeField(auto_now_add=True)
     updated_at = models.DateTimeField(auto_now=True)

     is_staff = models.BooleanField(default=False)
     is_admin = models.BooleanField(default=False)

     objects = UserManager()

     EMAIL_FIELD = "email"
     USERNAME_FIELD = "email"
     REQUIRED_FIELDS = ["company_name"]

     def save(self, *args, **kwargs):
         super(User, self).save(*args, **kwargs)
  4. Iterate over the payload dictionary and set all values to user object using setattr.
  5. Finally, save the user data to the database using user.save().
@router.post("/user", response={200: SuccessSchema, 400: ErrorSchema})
def superuser_post_user(request, data: CreateUserSchema):
    email = data.email

    if User.objects.filter(email=email).exists():
        return 400, {"message": "User with given email already exists"}

    user: User = User.objects.create()
    password = User.objects.make_random_password()
    user.set_password(password)
    for attr, value in data.dict().items():
        setattr(user, attr, value)
    user.save()

    return 200, {"message": "User creation success"}

At first glance, the function to handle this POST request seems to be correct. On the first hit to the endpoint, things went fine. But on any subsequent call, an error popped up

psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "account_user_email_key"
DETAIL: Key (email)=() already exists.

Issue

  • Without carefully looking at the error message, I jumped to the immediate conclusion that the user email might be a duplicate. But I was indeed checking if a User with the given email already exists or not.
  • On inspecting the database, I got to know that apart from the user we wanted to create, an extra user with all fields empty (apart from id field) is also created.
  • And somehow, on all subsequent calls to the API endpoint, this empty user creation is again called, but since we already have a user with an empty email, it invokes a violation of the unique constraint on email, explaining the error encountered above (DETAIL: Key (email)=() already exists.).

But when am I creating two users accidentally?

This is because of the way I have written the code. user: User = User.objects.create() creates an empty user with some id (the primary key). When I set different attributes from the payload dictionary and call user.save(), this user now has a different email, and hence the save method creates a new user with this actual user data. So in the back of my mind, I was hoping that the same user’s fields would get updated and stored in the database, which was not the case.

But I had NOT NULL constraint on email field. So shouldn’t creating a user with empty email not be allowed in the first place?

This was something that bugged me a lot. The line user: User = User.objects.create() should have thrown an error in the first place since email had null=False in the model class. On some googling, I got to know about this weird behaviour from here

No default Django behavior will save CHAR or TEXT types as Null - it will always use an empty string (‘’). null=False does not affect these types of fields. blank=False means that the field will be required by default when the model is used to render a ModelForm. It does not prevent you from manually saving a model instance without that value.

Avoid using null on string-based fields such as CharField and TextField. If a string-based field has null=True, that means it has two possible values for “no data”: NULL, and the empty string. In most cases, it’s redundant to have two possible values for “no data;” the Django convention is to use the empty string, not NULL. One exception is when a CharField has both unique=True and blank=True set. In this situation, null=True is required to avoid unique constraint violations when saving multiple objects with blank values.

Solution

Hence the correct modified code is as follows,

@router.post("/user", response={200: SuccessSchema, 400: ErrorSchema})
def superuser_post_user(request, data: CreateUserSchema):
    email = data.email

    if User.objects.filter(email=email).exists():
        return 400, {"message": "User with given email already exists"}

    user: User = User.objects.create(email=email)
    password = User.objects.make_random_password()
    user.set_password(password)
    for attr, value in data.dict(exclude={"email": ...}).items():
        setattr(user, attr, value)
    user.save()

    return 200, {"message": "User creation success"}

This can be further condensed as follows

@router.post("/user", response={200: SuccessSchema, 400: ErrorSchema})
def superuser_post_user(request, data: CreateUserSchema):
    email = data.email

    if User.objects.filter(email=email).exists():
        return 400, {"message": "User with given email already exists"}

    user: User = User.objects.create(**data.dict())
    password = User.objects.make_random_password()
    user.set_password(password)
    user.save()

    return 200, {"message": "User creation success"}