Django REST - FlexFields ======================== Flexible, dynamic fields and nested models for Django REST Framework serializers. Works with both Python 2 and 3. Overview ======== FlexFields (DRF-FF) for `Django REST Framework `__ is a package designed to provide a common baseline of functionality for dynamically setting fields and nested models within DRF serializers. To remove unneeded fields, you can dynamically set fields, including nested fields, via URL parameters ``(?fields=name,address.zip)`` or when configuring serializers. Additionally, you can dynamically expand fields from simple values to complex nested models, or treat fields as "deferred", and expand them on an as-needed basis. This package is designed for simplicity and provides two classes - a viewset class and a serializer class (or mixin) - with minimal magic and entanglement with DRF's foundational classes. Unless DRF makes significant changes to its serializers, you can count on this package to work (and if major changes are made, this package will be updated shortly thereafter). If you are familar with Django REST Framework, it shouldn't take you long to read over the code and see how it works. There are similar packages, such as the powerful `Dynamic REST `__, which does what this package does and more, but you may not need all those bells and whistles. There is also the more basic `Dynamic Fields Mixin `__, but it lacks functionality for field expansion and dot-notation field customiziation. Table of Contents: - `Installation <#installation>`__ - `Requirements <#requirements>`__ - `Basics <#basics>`__ - `Dynamic Field Expansion <#dynamic-field-expansion>`__ - `Deferred Fields <#deferred-fields>`__ - `Deep, Nested Expansion <#deep-nested-expansion>`__ - `Configuration from Serializer Options <#configuration-from-serializer-options>`__ - `Field Expansion on "List" Views <#field-expansion-on-list-views>`__ - `Use "~all" to Expand All Available Fields <#use-all-to-expand-all-available-fields>`__ - `Dynamically Setting Fields/Sparse Fieldsets <#dynamically-setting-fields>`__ - `From URL Parameters <#from-url-parameters>`__ - `From Serializer Options <#from-serializer-options>`__ - `Combining Dynamically-Set Fields and Field Expansion <#combining-dynamically-set-fields-and-field-expansion>`__ - `Serializer Introspection <#serializer-introspection>`__ - `Lazy evaluation of serializer <#lazy-evaluation-of-serializer>`__ - `Query optimization (experimental) <#query-optimization-experimental>`__ - `Change Log <#changelog>`__ - `Testing <#testing>`__ - `License <#license>`__ Installation ============ :: pip install drf-flex-fields Requirements ============ - Python >= 2.7 - Django >= 1.8 Basics ====== To use this package's functionality, your serializers need to subclass ``FlexFieldsModelSerializer`` or use the provided ``FlexFieldsSerializerMixin``. If you would like built-in protection for controlling when clients are allowed to expand resources when listing resource collections, your viewsets need to subclass ``FlexFieldsModelViewSet``. .. code:: python from rest_flex_fields import FlexFieldsModelViewSet, FlexFieldsModelSerializer class PersonViewSet(FlexFieldsModelViewSet): queryset = models.Person.objects.all() serializer_class = PersonSerializer # Whitelist fields that can be expanded when listing resources permit_list_expands = ['country'] class CountrySerializer(FlexFieldsModelSerializer): class Meta: model = Country fields = ('id', 'name', 'population') class PersonSerializer(FlexFieldsModelSerializer): class Meta: model = Person fields = ('id', 'name', 'country', 'occupation') expandable_fields = { 'country': (CountrySerializer, {'source': 'country'}) } Now you can make requests like ``GET /person?expand=country&fields=id,name,country`` to dynamically manipulate which fields are included, as well as expand primitive fields into nested objects. You can also use dot notation to control both the ``fields`` and ``expand`` settings at arbitrary levels of depth in your serialized responses. Read on to learn the details and see more complex examples. :heavy\_check\_mark: The examples below subclass ``FlexFieldsModelSerializer``, but the same can be accomplished by mixing in ``FlexFieldsSerializerMixin``, which is also importable from the same ``rest_flex_fields`` package. Dynamic Field Expansion ======================= To define an expandable field, add it to the ``expandable_fields`` within your serializer: .. code:: python class CountrySerializer(FlexFieldsModelSerializer): class Meta: model = Country fields = ['name', 'population'] class PersonSerializer(FlexFieldsModelSerializer): country = serializers.PrimaryKeyRelatedField(read_only=True) class Meta: model = Person fields = ['id', 'name', 'country', 'occupation'] expandable_fields = { 'country': (CountrySerializer, {'source': 'country', 'fields': ['name']}) } If the default serialized response is the following: .. code:: json { "id" : 13322, "name" : "John Doe", "country" : 12, "occupation" : "Programmer", } When you do a ``GET /person/13322?expand=country``, the response will change to: .. code:: json { "id" : 13322, "name" : "John Doe", "country" : { "name" : "United States" }, "occupation" : "Programmer", } Notice how ``population`` was ommitted from the nested ``country`` object. This is because ``fields`` was set to ``['name']`` when passed to the embedded ``CountrySerializer``. You will learn more about this later on. Deferred Fields --------------- Alternatively, you could treat ``country`` as a "deferred" field by not defining it among the default fields. To make a field deferred, only define it within the serializer's ``expandable_fields``. Deep, Nested Expansion ---------------------- Let's say you add ``StateSerializer`` as serializer nested inside the country serializer above: .. code:: python class StateSerializer(FlexFieldsModelSerializer): class Meta: model = State fields = ['name', 'population'] class CountrySerializer(FlexFieldsModelSerializer): class Meta: model = Country fields = ['name', 'population'] expandable_fields = { 'states': (StateSerializer, {'source': 'states', 'many': True}) } class PersonSerializer(FlexFieldsModelSerializer): country = serializers.PrimaryKeyRelatedField(read_only=True) class Meta: model = Person fields = ['id', 'name', 'country', 'occupation'] expandable_fields = { 'country': (CountrySerializer, {'source': 'country', 'fields': ['name']}) } Your default serialized response might be the following for ``person`` and ``country``, respectively: .. code:: json { "id" : 13322, "name" : "John Doe", "country" : 12, "occupation" : "Programmer", } { "id" : 12, "name" : "United States", "states" : "http://www.api.com/countries/12/states" } But if you do a ``GET /person/13322?expand=country.states``, it would be: .. code:: json { "id" : 13322, "name" : "John Doe", "occupation" : "Programmer", "country" : { "id" : 12, "name" : "United States", "states" : [ { "name" : "Ohio", "population": 11000000 } ] } } Please be kind to your database, as this could incur many additional queries. Though, you can mitigate this impact through judicious use of ``prefetch_related`` and ``select_related`` when defining the queryset for your viewset. Configuration from Serializer Options ------------------------------------- You could accomplish the same result (expanding the ``states`` field within the embedded country serializer) by explicitly passing the ``expand`` option within your serializer: .. code:: python class PersonSerializer(FlexFieldsModelSerializer): class Meta: model = Person fields = ['id', 'name', 'country', 'occupation'] expandable_fields = { 'country': (CountrySerializer, {'source': 'country', 'expand': ['states']}) } Field Expansion on "List" Views ------------------------------- By default, when subclassing ``FlexFieldsModelViewSet``, you can only expand fields when you are retrieving single resources, in order to protect yourself from careless clients. However, if you would like to make a field expandable even when listing collections, you can add the field's name to the ``permit_list_expands`` property on the viewset. Just make sure you are wisely using ``select_related`` and ``prefetch_related`` in the viewset's queryset. You can take advantage of a utility function, ``is_expanded()`` to adjust the queryset accordingly. Example: .. code:: python from drf_flex_fields import is_expanded class PersonViewSet(FlexFieldsModelViewSet): permit_list_expands = ['employer'] serializer_class = PersonSerializer def get_queryset(self): queryset = models.Person.objects.all() if is_expanded(self.request, 'employer'): queryset = queryset.select_related('employer') return queryset Use "~all" to Expand All Available Fields ----------------------------------------- You can set ``expand=~all`` to automatically expand all fields that are available for expansion. This will take effect only for the top-level serializer; if you need to also expand fields that are present on deeply nested models, then you will need to explicitly pass their values using dot notation. Dynamically Setting Fields (Sparse Fields) ========================================== You can use either they ``fields`` or ``omit`` keywords to declare only the fields you want to include or to specify fields that should be excluded. From URL Parameters ------------------- You can dynamically set fields, with the configuration originating from the URL parameters or serializer options. Consider this as a default serialized response: .. code:: json { "id" : 13322, "name" : "John Doe", "country" : { "name" : "United States", "population": 330000000 }, "occupation" : "Programmer", "hobbies" : ["rock climbing", "sipping coffee"] } To whittle down the fields via URL parameters, simply add ``?fields=id,name,country`` to your requests to get back: .. code:: json { "id" : 13322, "name" : "John Doe", "country" : { "name" : "United States", "population: 330000000 } } Or, for more specificity, you can use dot-notation, ``?fields=id,name,country.name``: .. code:: json { "id" : 13322, "name" : "John Doe", "country" : { "name" : "United States", } } Or, if you want to leave out the nested country object, do ``?omit=country``: .. code:: json { "id" : 13322, "name" : "John Doe", "occupation" : "Programmer", "hobbies" : ["rock climbing", "sipping coffee"] } From Serializer Options ----------------------- You could accomplish the same outcome as the example above by passing options to your serializers. With this approach, you lose runtime dynamism, but gain the ability to re-use serializers, rather than creating a simplified copy of a serializer for the purposes of embedding it. The example below uses the ``fields`` keyword, but you can also pass in keyword argument for ``omit`` to exclude specific fields. .. code:: python from rest_flex_fields import FlexFieldsModelSerializer class CountrySerializer(FlexFieldsModelSerializer): class Meta: model = Country fields = ['id', 'name', 'population'] class PersonSerializer(FlexFieldsModelSerializer): country: CountrySerializer(fields=['name']) class Meta: model = Person fields = ['id', 'name', 'country', 'occupation', 'hobbies'] serializer = PersonSerializer(person, fields=["id", "name", "country.name"]) print(serializer.data) >>>{ "id": 13322, "name": "John Doe", "country": { "name": "United States", } } Combining Dynamically Set Fields and Field Expansion ==================================================== You may be wondering how things work if you use both the ``expand`` and ``fields`` option, and there is overlap. For example, your serialized person model may look like the following by default: .. code:: json { "id": 13322, "name": "John Doe", "country": { "name": "United States", } } However, you make the following request ``HTTP GET /person/13322?include=id,name&expand=country``. You will get the following back: .. code:: json { "id": 13322, "name": "John Doe" } The ``include`` field takes precedence over ``expand``. That is, if a field is not among the set that is explicitly alllowed, it cannot be expanded. If such a conflict occurs, you will not pay for the extra database queries - the expanded field will be silently abandoned. Serializer Introspection ======================== When using an instance of ``FlexFieldsModelSerializer``, you can examine the property ``expanded_fields`` to discover which fields, if any, have been dynamically expanded. Lazy evaluation of serializer ============================= If you want to lazily evaluate the reference to your nested serializer class from a string inside expandable\_fields, you need to use this syntax: .. code:: python expandable_fields = { 'record_set': ('.RelatedSerializer', {'source': 'related_set', 'many': True}) } Substitute the name of your Django app where the serializer is found for ````. This allows to reference a serializer that has not yet been defined. Query optimization (experimental) ================================= An experimental filter backend is available to help you automatically reduce the number of SQL queries and their transfer size. *This feature has not been tested thorougly and any help testing and reporting bugs is greatly appreciated.* You can add FlexFieldFilterBackend to ``DEFAULT_FILTER_BACKENDS`` in the settings: .. code:: python # settings.py REST_FRAMEWORK = { 'DEFAULT_FILTER_BACKENDS': ( 'rest_flex_fields.filter_backends.FlexFieldsFilterBackend', # ... ), # ... } It will automatically call ``select_related`` and ``prefetch_related`` on the current QuerySet by determining which fields are needed from many-to-many and foreign key-related models. For sparse fields requests (``?omit=fieldX,fieldY`` or ``?fields=fieldX,fieldY``), the backend will automatically call ``only(*field_names)`` using only the fields needed for serialization. **WARNING:** The optimization currently works only for one nesting level. Changelog ========== 0.6.0 (May 2019) ---------------- - Adds experimental support for automatically SQL query optimization via a ``FlexFieldsFilterBackend``. Thanks ADR-007! - Adds CircleCI config file. Thanks mikeIFTS! - Moves declaration of ``expandable_fields`` to ``Meta`` class on serialzer for consistency with DRF (will continue to support declaration as class property) 0.5.0 (April 2019) ------------------ - Added support for ``omit`` keyword for field exclusion. Code clean up and improved test coverage. 0.3.4 (May 2018) ---------------- - Handle case where ``request`` is ``None`` when accessing request object from serializer. Thanks @jsatt! 0.3.3 (April 2018) ------------------ - Exposes ``FlexFieldsSerializerMixin`` in addition to ``FlexFieldsModelSerializer``. Thanks @jsatt! Testing ======= Tests are found in a simplified DRF project in the ``/tests`` folder. Install the project requirements and do ``./manage.py test`` to run them. License ======= See `License `__.