django/docs/releases/4.0.txt

============================================
Django 4.0 release notes - UNDER DEVELOPMENT
============================================

*Expected December 2021*

Welcome to Django 4.0!

These release notes cover the :ref:`new features <whats-new-4.0>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-4.0>` you'll
want to be aware of when upgrading from Django 3.2 or earlier. We've
:ref:`begun the deprecation process for some features
<deprecated-features-4.0>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 4.0 supports Python 3.8, 3.9, and 3.10. We **highly recommend** and only
officially support the latest release of each series.

The Django 3.2.x series is the last to support Python 3.6 and 3.7.

.. _whats-new-4.0:

What's new in Django 4.0
========================

Functional unique constraints
-----------------------------

The new :attr:`*expressions <django.db.models.UniqueConstraint.expressions>`
positional argument of
:class:`UniqueConstraint() <django.db.models.UniqueConstraint>` enables
creating functional unique constraints on expressions and database functions.
For example::

    from django.db import models
    from django.db.models import UniqueConstraint
    from django.db.models.functions import Lower


    class MyModel(models.Model):
        first_name = models.CharField(max_length=255)
        last_name = models.CharField(max_length=255)

        class Meta:
            indexes = [
                UniqueConstraint(
                    Lower('first_name'),
                    Lower('last_name').desc(),
                    name='first_last_name_unique',
                ),
            ]

Functional unique constraints are added to models using the
:attr:`Meta.constraints <django.db.models.Options.constraints>` option.

``scrypt`` password hasher
--------------------------

The new :ref:`scrypt password hasher <scrypt-usage>` is more secure and
recommended over PBKDF2. However, it's not the default as it requires OpenSSL
1.1+ and more memory.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The ``admin/base.html`` template now has a new block ``header`` which
  contains the admin site header.

* The new :meth:`.ModelAdmin.get_formset_kwargs` method allows customizing the
  keyword arguments passed to the constructor of a formset.

* The navigation sidebar now has a quick filter toolbar.

* The new context variable ``model`` which contains the model class for each
  model is added to the :meth:`.AdminSite.each_context` method.

* The new :attr:`.ModelAdmin.search_help_text` attribute allows specifying a
  descriptive text for the search box.

:mod:`django.contrib.admindocs`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The admindocs now allows esoteric setups where :setting:`ROOT_URLCONF` is not
  a string.

* The model section of the ``admindocs`` now shows cached properties.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased from
  260,000 to 320,000.

* The new
  :attr:`LoginView.next_page <django.contrib.auth.views.LoginView.next_page>`
  attribute and
  :meth:`~django.contrib.auth.views.LoginView.get_default_redirect_url` method
  allow customizing the redirect after login.

:mod:`django.contrib.contenttypes`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Added support for SpatiaLite 5.

* :class:`~django.contrib.gis.gdal.GDALRaster` now allows creating rasters in
  any GDAL virtual filesystem.

:mod:`django.contrib.messages`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The PostgreSQL backend now supports connecting by a service name. See
  :ref:`postgresql-connection-settings` for more details.

* The new :class:`~django.contrib.postgres.operations.AddConstraintNotValid`
  operation allows creating check constraints on PostgreSQL without verifying
  that all existing rows satisfy the new constraint.

* The new :class:`~django.contrib.postgres.operations.ValidateConstraint`
  operation allows validating check constraints which were created using
  :class:`~django.contrib.postgres.operations.AddConstraintNotValid` on
  PostgreSQL.

* The new
  :class:`ArraySubquery() <django.contrib.postgres.expressions.ArraySubquery>`
  expression allows using subqueries to construct lists of values on
  PostgreSQL.

:mod:`django.contrib.redirects`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

:mod:`django.contrib.sitemaps`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

:mod:`django.contrib.sites`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage` now
  replaces paths to JavaScript source map references with their hashed
  counterparts.

* :class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage` now
  replaces paths to JavaScript modules in ``import`` and ``export`` statements
  with their hashed counterparts.

* The new ``manifest_storage`` argument of
  :class:`~django.contrib.staticfiles.storage.ManifestFilesMixin` and
  :class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage`
  allows customizing the manifest file storage.

:mod:`django.contrib.syndication`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* ...

Cache
~~~~~

* The new async API for ``django.core.cache.backends.base.BaseCache`` begins
  the process of making cache backends async-compatible. The new async methods
  all have ``a`` prefixed names, e.g. ``aadd()``, ``aget()``, ``aset()``,
  ``aget_or_set()``, or ``adelete_many()``.

  Going forward, the ``a`` prefix will be used for async variants of methods
  generally.

CSRF
~~~~

* CSRF protection now consults the ``Origin`` header, if present. To facilitate
  this, :ref:`some changes <csrf-trusted-origins-changes-4.0>` to the
  :setting:`CSRF_TRUSTED_ORIGINS` setting are required.

Decorators
~~~~~~~~~~

* ...

Email
~~~~~

* ...

Error Reporting
~~~~~~~~~~~~~~~

* ...

File Storage
~~~~~~~~~~~~

* ...

File Uploads
~~~~~~~~~~~~

* ...

Forms
~~~~~

* :class:`~django.forms.ModelChoiceField` now includes the provided value in
  the ``params`` argument of a raised
  :exc:`~django.core.exceptions.ValidationError` for the ``invalid_choice``
  error message. This allows custom error messages to use the ``%(value)s``
  placeholder.

* :class:`~django.forms.formsets.BaseFormSet` now renders non-form errors with
  an additional class of ``nonform`` to help distinguish them from
  form-specific errors.

* :class:`~django.forms.formsets.BaseFormSet` now allows customizing the widget
  used when deleting forms via
  :attr:`~django.forms.formsets.BaseFormSet.can_delete` by setting the
  :attr:`~django.forms.formsets.BaseFormSet.deletion_widget` attribute or
  overriding :meth:`~django.forms.formsets.BaseFormSet.get_deletion_widget`
  method.

Generic Views
~~~~~~~~~~~~~

* :class:`~django.views.generic.edit.DeleteView` now uses
  :class:`~django.views.generic.edit.FormMixin`, allowing you to provide a
  :class:`~django.forms.Form` subclass, with a checkbox for example, to confirm
  deletion. In addition, this allows ``DeleteView`` to function with
  :class:`django.contrib.messages.views.SuccessMessageMixin`.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* ...

Logging
~~~~~~~

* The alias of the database used in an SQL call is now passed as extra context
  along with each message to the :ref:`django-db-logger` logger.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The :djadmin:`runserver` management command now supports the
  :option:`--skip-checks` option.

* On PostgreSQL, :djadmin:`dbshell` now supports specifying a password file.

* The :djadmin:`shell` command now respects :py:data:`sys.__interactivehook__`
  at startup. This allows loading shell history between interactive sessions.
  As a consequence, ``readline`` is no longer loaded if running in *isolated*
  mode.

* The new :attr:`BaseCommand.suppressed_base_arguments
  <django.core.management.BaseCommand.suppressed_base_arguments>` attribute
  allows suppressing unsupported default command options in the help output.

* The new :option:`startapp --exclude` and :option:`startproject --exclude`
  options allow excluding directories from the template.

Migrations
~~~~~~~~~~

* ...

Models
~~~~~~

* New :meth:`QuerySet.contains(obj) <.QuerySet.contains>` method returns
  whether the queryset contains the given object. This tries to perform the
  query in the simplest and fastest way possible.

* The new ``precision`` argument of the
  :class:`Round() <django.db.models.functions.Round>` database function allows
  specifying the number of decimal places after rounding.

* :meth:`.QuerySet.bulk_create` now sets the primary key on objects when using
  SQLite 3.35+.

* :class:`~django.db.models.DurationField` now supports multiplying and
  dividing by scalar values on SQLite.

* :meth:`.QuerySet.bulk_update` now returns the number of objects updated.

* The new :attr:`.Aggregate.empty_aggregate_value` attribute allows specifying
  a value to return when the aggregation is used over an empty result set.

* The ``skip_locked`` argument of :meth:`.QuerySet.select_for_update()` is now
  allowed on MariaDB 10.6+.

* :class:`~django.db.models.Lookup` expressions may now be used in ``QuerySet``
  annotations, aggregations, and directly in filters.

* The new :ref:`default <aggregate-default>` argument for built-in aggregates
  allows specifying a value to be returned when the queryset (or grouping)
  contains no entries, rather than ``None``.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* The :class:`~django.middleware.security.SecurityMiddleware` now adds the
  :ref:`Cross-Origin Opener Policy <cross-origin-opener-policy>` header with a
  value of ``'same-origin'`` to prevent cross-origin popups from sharing the
  same browsing context. You can prevent this header from being added by
  setting the :setting:`SECURE_CROSS_ORIGIN_OPENER_POLICY` setting to ``None``.

Security
~~~~~~~~

* ...

Serialization
~~~~~~~~~~~~~

* ...

Signals
~~~~~~~

* The new ``stdout`` argument for :func:`~django.db.models.signals.pre_migrate`
  and :func:`~django.db.models.signals.post_migrate` signals allows redirecting
  output to a stream-like object. It should be preferred over
  :py:data:`sys.stdout` and :py:func:`print` when emitting verbose output in
  order to allow proper capture when testing.

Templates
~~~~~~~~~

* ...

Tests
~~~~~

* The new ``serialized_aliases`` argument of
  :func:`django.test.utils.setup_databases` determines which
  :setting:`DATABASES` aliases test databases should have their state
  serialized to allow usage of the
  :ref:`serialized_rollback <test-case-serialized-rollback>` feature.

* Django test runner now supports a :option:`--buffer <test --buffer>` option
  with parallel tests.

* The new ``logger`` argument to :class:`~django.test.runner.DiscoverRunner`
  allows a Python :py:ref:`logger <logger>` to be used for logging.

* The new :meth:`.DiscoverRunner.log` method provides a way to log messages
  that uses the ``DiscoverRunner.logger``, or prints to the console if not set.

* Django test runner now supports a :option:`--shuffle <test --shuffle>` option
  to execute tests in a random order.

* The :option:`test --parallel` option now supports the value ``auto`` to run
  one test process for each processor core.

* :meth:`.TestCase.captureOnCommitCallbacks` now captures new callbacks added
  while executing :func:`.transaction.on_commit` callbacks.

URLs
~~~~

* ...

Utilities
~~~~~~~~~

* ...

Validators
~~~~~~~~~~

* ...

.. _backwards-incompatible-4.0:

Backwards incompatible changes in 4.0
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* ``DatabaseOperations.year_lookup_bounds_for_date_field()`` and
  ``year_lookup_bounds_for_datetime_field()`` methods now take the optional
  ``iso_year`` argument in order to support bounds for ISO-8601 week-numbering
  years.

* The second argument of ``DatabaseSchemaEditor._unique_sql()`` and
  ``_create_unique_sql()`` methods is now fields instead of columns.

:mod:`django.contrib.gis`
-------------------------

* Support for PostGIS 2.3 is removed.

* Support for GDAL 2.0 and GEOS 3.5 is removed.

Dropped support for PostgreSQL 9.6
----------------------------------

Upstream support for PostgreSQL 9.6 ends in November 2021. Django 4.0 supports
PostgreSQL 10 and higher.

Dropped support for Oracle 12.2 and 18c
---------------------------------------

Upstream support for Oracle 12.2 ends in March 2022 and for Oracle 18c it ends
in June 2021. Django 3.2 will be supported until April 2024. Django 4.0
officially supports Oracle 19c.

.. _csrf-trusted-origins-changes-4.0:

``CSRF_TRUSTED_ORIGINS`` changes
--------------------------------

Format change
~~~~~~~~~~~~~

Values in the :setting:`CSRF_TRUSTED_ORIGINS` setting must include the scheme
(e.g. ``'http://'`` or ``'https://'``) instead of only the hostname.

Also, values that started with a dot, must now also include an asterisk before
the dot. For example, change ``'.example.com'`` to ``'https://*.example.com'``.

A system check detects any required changes.

Configuring it may now be required
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As CSRF protection now consults the ``Origin`` header, you may need to set
:setting:`CSRF_TRUSTED_ORIGINS`, particularly if you allow requests from
subdomains by setting :setting:`CSRF_COOKIE_DOMAIN` (or
:setting:`SESSION_COOKIE_DOMAIN` if :setting:`CSRF_USE_SESSIONS` is enabled) to
a value starting with a dot.

``SecurityMiddleware`` no longer sets the ``X-XSS-Protection`` header
---------------------------------------------------------------------

The :class:`~django.middleware.security.SecurityMiddleware` no longer sets the
``X-XSS-Protection`` header if the ``SECURE_BROWSER_XSS_FILTER`` setting is
``True``. The setting is removed.

Most modern browsers don't honor the ``X-XSS-Protection`` HTTP header. You can
use Content-Security-Policy_ without allowing ``'unsafe-inline'`` scripts
instead.

If you want to support legacy browsers and set the header, use this line in a
custom middleware::

    response.headers.setdefault('X-XSS-Protection', '1; mode=block')

.. _Content-Security-Policy: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy

Migrations autodetector changes
-------------------------------

The migrations autodetector now uses model states instead of model classes.
Also, migration operations for ``ForeignKey`` and ``ManyToManyField`` fields no
longer specify attributes which were not passed to the fields during
initialization.

As a side-effect, running ``makemigrations`` might generate no-op
``AlterField`` operations for ``ManyToManyField`` and ``ForeignKey`` fields in
some cases.

Miscellaneous
-------------

* Support for ``cx_Oracle`` < 7.0 is removed.

* To allow serving a Django site on a subpath without changing the value of
  :setting:`STATIC_URL`, the leading slash is removed from that setting (now
  ``'static/'``) in the default :djadmin:`startproject` template.

* The :class:`~django.contrib.admin.AdminSite` method for the admin ``index``
  view is no longer decorated with ``never_cache`` when accessed directly,
  rather than via the recommended ``AdminSite.urls`` property, or
  ``AdminSite.get_urls()`` method.

* Unsupported operations on a sliced queryset now raise ``TypeError`` instead
  of ``AssertionError``.

* The undocumented ``django.test.runner.reorder_suite()`` function is renamed
  to ``reorder_tests()``. It now accepts an iterable of tests rather than a
  test suite, and returns an iterator of tests.

* Calling ``FileSystemStorage.delete()`` with an empty ``name`` now raises
  ``ValueError`` instead of ``AssertionError``.

* Calling ``EmailMultiAlternatives.attach_alternative()`` or
  ``EmailMessage.attach()`` with an invalid ``content`` or ``mimetype``
  arguments now raise ``ValueError`` instead of ``AssertionError``.

* :meth:`~django.test.SimpleTestCase.assertHTMLEqual` no longer considers a
  non-boolean attribute without a value equal to an attribute with the same
  name and value.

* Tests that fail to load, for example due to syntax errors, now always match
  when using :option:`test --tag`.

* The undocumented ``django.contrib.admin.utils.lookup_needs_distinct()``
  function is renamed to ``lookup_spawns_duplicates()``.

* The undocumented ``HttpRequest.get_raw_uri()`` method is removed. The
  :meth:`.HttpRequest.build_absolute_uri` method may be a suitable alternative.

* The ``object`` argument of undocumented ``ModelAdmin.log_addition()``,
  ``log_change()``, and ``log_deletion()`` methods is renamed to ``obj``.

* :class:`~django.utils.feedgenerator.RssFeed`,
  :class:`~django.utils.feedgenerator.Atom1Feed`, and their subclasses now
  emit elements with no content as self-closing tags.

* ``NodeList.render()`` no longer casts the output of ``render()`` method for
  individual nodes to a string. ``Node.render()`` should always return a string
  as documented.

* The ``where_class`` property of ``django.db.models.sql.query.Query`` and the
  ``where_class`` argument to the private ``get_extra_restriction()`` method of
  ``ForeignObject`` and ``ForeignObjectRel`` are removed. If needed, initialize
  ``django.db.models.sql.where.WhereNode`` instead.

* The ``filter_clause`` argument of the undocumented ``Query.add_filter()``
  method is replaced by two positional arguments ``filter_lhs`` and
  ``filter_rhs``.

* :class:`~django.middleware.csrf.CsrfViewMiddleware` now uses
  ``request.META['CSRF_COOKIE_NEEDS_UPDATE']`` in place of
  ``request.META['CSRF_COOKIE_USED']``, ``request.csrf_cookie_needs_reset``,
  and ``response.csrf_cookie_set`` to track whether the CSRF cookie should be
  sent. This is an undocumented, private API.

* The undocumented ``TRANSLATOR_COMMENT_MARK`` constant is moved from
  ``django.template.base`` to ``django.utils.translation.template``.

* The ``real_apps`` argument of the undocumented
  ``django.db.migrations.state.ProjectState.__init__()`` method must now be a
  set if provided.

* :class:`~django.forms.RadioSelect` and
  :class:`~django.forms.CheckboxSelectMultiple` widgets are now rendered in
  ``<div>`` tags so they are announced more concisely by screen readers. If you
  need the previous behavior, :ref:`override the widget template
  <overriding-built-in-widget-templates>` with the appropriate template from
  Django 3.2.

.. _deprecated-features-4.0:

Features deprecated in 4.0
==========================

Time zone support
-----------------

In order to follow good practice, the default value of the :setting:`USE_TZ`
setting will change from ``False`` to ``True``, and time zone support will be
enabled by default, in Django 5.0.

Note that the default :file:`settings.py` file created by
:djadmin:`django-admin startproject <startproject>` includes
:setting:`USE_TZ = True <USE_TZ>` since Django 1.4.

You can set ``USE_TZ`` to ``False`` in your project settings before then to
opt-out.

Miscellaneous
-------------

* ``SERIALIZE`` test setting is deprecated as it can be inferred from the
  :attr:`~django.test.TestCase.databases` with the
  :ref:`serialized_rollback <test-case-serialized-rollback>` option enabled.

* The undocumented ``django.utils.baseconv`` module is deprecated.

* The undocumented ``django.utils.datetime_safe`` module is deprecated.

* The default sitemap protocol for sitemaps built outside the context of a
  request will change from ``'http'`` to ``'https'`` in Django 5.0.

* The ``extra_tests`` argument for :meth:`.DiscoverRunner.build_suite` and
  :meth:`.DiscoverRunner.run_tests` is deprecated.

* The :class:`~django.contrib.postgres.aggregates.ArrayAgg`,
  :class:`~django.contrib.postgres.aggregates.JSONBAgg`, and
  :class:`~django.contrib.postgres.aggregates.StringAgg` aggregates will return
  ``None`` when there are no rows instead of ``[]``, ``[]``, and ``''``
  respectively in Django 5.0. If you need the previous behavior, explicitly set
  ``default`` to ``Value([])``, ``Value('[]')``, or ``Value('')``.

Features removed in 4.0
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 4.0.

See :ref:`deprecated-features-3.0` for details on these changes, including how
to remove usage of these features.

* ``django.utils.http.urlquote()``, ``urlquote_plus()``, ``urlunquote()``, and
  ``urlunquote_plus()`` are removed.

* ``django.utils.encoding.force_text()`` and ``smart_text()`` are removed.

* ``django.utils.translation.ugettext()``, ``ugettext_lazy()``,
  ``ugettext_noop()``, ``ungettext()``, and ``ungettext_lazy()`` are removed.

* ``django.views.i18n.set_language()`` doesn't set the user language in
  ``request.session`` (key ``_language``).

* ``alias=None`` is required in the signature of
  ``django.db.models.Expression.get_group_by_cols()`` subclasses.

* ``django.utils.text.unescape_entities()`` is removed.

* ``django.utils.http.is_safe_url()`` is removed.

See :ref:`deprecated-features-3.1` for details on these changes, including how
to remove usage of these features.

* The ``PASSWORD_RESET_TIMEOUT_DAYS`` setting is removed.

* The :lookup:`isnull` lookup no longer allows using non-boolean values as the
  right-hand side.

* The ``django.db.models.query_utils.InvalidQuery`` exception class is removed.

* The ``django-admin.py`` entry point is removed.

* The ``HttpRequest.is_ajax()`` method is removed.

* Support for the pre-Django 3.1 encoding format of cookies values used by
  ``django.contrib.messages.storage.cookie.CookieStorage`` is removed.

* Support for the pre-Django 3.1 password reset tokens in the admin site (that
  use the SHA-1 hashing algorithm) is removed.

* Support for the pre-Django 3.1 encoding format of sessions is removed.

* Support for the pre-Django 3.1 ``django.core.signing.Signer`` signatures
  (encoded with the SHA-1 algorithm) is removed.

* Support for the pre-Django 3.1 ``django.core.signing.dumps()`` signatures
  (encoded with the SHA-1 algorithm) in ``django.core.signing.loads()`` is
  removed.

* Support for the pre-Django 3.1 user sessions (that use the SHA-1 algorithm)
  is removed.

* The ``get_response`` argument for
  ``django.utils.deprecation.MiddlewareMixin.__init__()`` is required and
  doesn't accept ``None``.

* The ``providing_args`` argument for ``django.dispatch.Signal`` is removed.

* The ``length`` argument for ``django.utils.crypto.get_random_string()`` is
  required.

* The ``list`` message for ``ModelMultipleChoiceField`` is removed.

* Support for passing raw column aliases to ``QuerySet.order_by()`` is removed.

* The ``NullBooleanField`` model field is removed, except for support in
  historical migrations.

* ``django.conf.urls.url()`` is removed.

* The ``django.contrib.postgres.fields.JSONField`` model field is removed,
  except for support in historical migrations.

* ``django.contrib.postgres.fields.jsonb.KeyTransform`` and
  ``django.contrib.postgres.fields.jsonb.KeyTextTransform`` are removed.

* ``django.contrib.postgres.forms.JSONField`` is removed.

* The ``{% ifequal %}`` and ``{% ifnotequal %}`` template tags are removed.

* The ``DEFAULT_HASHING_ALGORITHM`` transitional setting is removed.