Re-using a file in multiple serializers in the same request

In a Django Rest Framework project, an uploaded file is intended to be used once, when the model instance is being created by the serializer. After DRF completes its create flow, the InMemoryUploadedFile object has its closed property set to True and there is no way to re-open that file.

If we try to use that same file, i.e.: an image, to create another instance of some model using a serializer in that same request, we’ll receive an IOError exception telling us that we’re trying to manipulate a file that's already closed.

To better illustrate this, let’s use this simplified excerpt of a method from our codebase.

class PostTemplateSerializer(serializers.ModelSerializer):
  def create(self, validated_data):
    ad_template_data = self._prepare_ad_template_serializer_data(validated_data)
    post_template = super().create(validated_data)
    ad_template_serializer = AdTemplateSerializer(data=ad_template_data)
    ad_template_serializer.is_valid(raise_exception=True)
    ad_template_serializer.save()
    return post_template

The PostTemplateSerializer main purpose is to create PostTemplate instances. However, we wanted to expand its functionality to allow us to create a “copy” of it, an AdTemplate which holds some data that was used to create the post template and will be able to be re-used to create more post templates in the future, including the post template main image.

To avoid hitting the endpoint multiple times and to ensure data consistency (i.e.: if the ad template creation fails for some reason we should also rollback the creation of the post template), we’ve opted to bake this new functionality in the same endpoint.

Doing so, however, made us face the IOError multiple times when trying to use the same image in a post template and in an ad template. In order to solve that, we’ve made the following attempts:

1. Use the image file from validated_data in ad_template_data

The first approach was the one that caused the error in the first place. Inside _prepare_ad_template_serializer_data we’ve copied validated_data["post_image"] into ad_template_data["post_image"].

Since the image is an object, its reference remained shared by both dictionaries, meaning that when the validated_data image is closed after line 4, the object on ad_template_data would also close, raising the IOError exception when we try to save the ad template object.

2. Use the post_template instance image in the ad template serializer

This idea consisted of moving the ad template creation outside of the serializer. This wouldn’t be a issue since the campaign creation view has the @transaction.atomic decorator, so we weren’t at the risk of having inconsistencies (i.e.: if the campaign was created, but there was an error in the ad template, there’d be a rollback, as intended).

First, we started by creating the post template instance and then access the object properties and get the image file from it. This image file would then be passed to the ad template serializer via the data parameter. This worked during our manual tests and it was supposed to be our solution. However, when we started running the automated tests with Percy, the flow started breaking due to the same IOError.

3. Make a copy of the uploaded file before using it

The solution we ended up using was to create an in-memory copy of the uploaded image file while it was still usable (i.e.: before being used by the campaign serializer). Since the copy was a fully new object, it wouldn’t share any reference with the original image, so it wouldn’t become closed at the same time as the original. Below is a code snippet of the implementation we’ve made with some comments in there.

def make_image_copy(image, preffix = "copy-"):
    # Ensure the original image file will be at the "start" before we read its contents
    image.file.seek(0)
    img_file_bytes = io.BytesIO(image.file.read())
    img_format = guess_extension(img_file_bytes)
    # Ensure all images (original and copy) will have their
    # file pointers placed at the start of the content
    image.file.seek(0)
    img_file_bytes.seek(0)
    return InMemoryUploadedFile(
        img_file_bytes,
        None,
        f"{preffix}{image.name}",
        f"image/{img_format}",
        sys.getsizeof(img_file_bytes),
        None,
    )