This last article in the series will focus on the steps necessary to upgrade Django and how to prepare for the future. Except for a single section dedicated to Django 1.7, the content in this article applies to any Django version. Advanced developers may wish to jump directly to Upgrading Django to 1.7.

The article will start with how to upgrade Django and then look at why you and your organization should upgrade. Finally, we will look to the future to see what Django has in store for us. At the end of the article you will find a link to a list of all the relevant links used in this series as well as an upgrade checklist.

Upgrading Django

Sir Lancelot: You were in terrible peril.

Sir Galahad: Look, let me go back in there and face the peril.

Sir Lancelot: No, it's too perilous.

Upgrading Django can be a daunting task. Veterans have described the process as a Kafkaesque nightmare, and beginners sometimes avoid the task entirely. The challenge may seem especially fearsome when it involves upgrading much older versions of Django or if the project is particularly complex.

Upgrading Django should not inspire the fear it currently does. The act of upgrading is a tedious process---given the right setup---but it is not a difficult one. This article will make upgrading Django a truly boring experience for you.

Despite the monotony of the process, the payoff of upgrading makes the investment well worth the effort. This article will start by providing you with the tools and setup you need to make upgrading easy. We will then break down the process into small, manageable steps for you to follow. Even if a project can't get all the way there at once, taking even a few of the steps outlined will save you time in the long run and is better than giving up. With the tools and process established, we will use our knowledge to outline an upgrade to Django 1.7. Finally, we will discuss reasons for upgrading and the future of Django.

Tools For Upgrading Django

This section seeks to outline the tools you may use to upgrade Django. We will:

1. discuss the documentation available to you
2. discuss the utility of testing suites and recommended testing calls
3. outline helpful upgrade environments.

Tools in Django's Documentation

Django outlines upgrade resources in the documentation. There are three notable resources:

1. Django Versioning and Release Process
2. Django Release Notes
3. Django Deprecation Timeline

If you read Part I's Django Versioning System section, then you will already be familiar with the docs for Django's Release Process (#1).

The Django release notes (#2) are documents released for every (major and minor) release of Django. Each set of notes lists new features, bug fixes, and deprecated features. The release notes are essential when upgrading any release. Take time to read through them and identify the key changes that must occur in your project. The "Backwards-Incompatible Changes" and "Features Deprecated" sections are particularly helpful and should be where you start. I will refer to them throughout this article.

The "Backwards-Incompatible Changes" section has been available in release notes since Django 1.0. The "Features Deprecated" section has been available in release notes since Django 1.2.

The deprecation timeline (#3) lists when features will be completely removed from Django. It is possible for Django to raise some confusing errors if a project has continued to use a deprecated feature. What's more, finding a deprecated feature that has warnings suppressed can prove tricky. The timeline may help you make sense of these errors and locate the problem. Think of the deprecation timeline an anatomy manual for those times when you discover dinosaur bones in your backyard.

Testing Suite

Tests are vital to the upgrade process. At their most basic, tests allow developers to check behavior at the time they program a feature. However, tests also assure you that you maintain expected behavior as you make changes to your code. When upgrading Django, you want to isolate the changes you are making: you are changing code without changing user behavior. Without tests, you must manually check site behavior every time you commit a change, which is time consuming. Tests are thus the only efficient method to use to ensure you achieve this goal.

The topic of writing tests is beyond the scope of this article. If you do not have tests for your project, or are looking to improve your test suite, I recommend Harry Percival's "Test-Driven Development with Python" book. Harry's book is available online for free or for purchase in various formats from O'Reilly.

As you upgrade Django (or make any change to a project, for that matter), remember to test regularly and to verify the coverage of your tests. Doing so will make it immediately obvious which change caused any potential issues. If you wait to test until after you've made many changes or upgraded several dependencies, the change that caused the error may not be as obvious. The Django upgrade instructions recommend invoking tests with -Wall, which causes Python to "print a warning each time it occurs." Aymeric Augustin also pointed out the use of -Werror, which causes Python to "raise an exception instead of printing a warning message."

In conclusion, a test suite is necessary to upgrading a project of any complexity. Testing will save you time and provide a confidence metric for how your changes have affected your website.

Controlled Environment

Creating a controlled, dedicated environment for development and testing can help you during the upgrade process.

Locally, making use of virtualenv (and virtualenvwrapper) can make your life much easier, as they will allow you to move between development environments as needed. If you are not currently using a Django-specific environment, take a look at Jeff Knupp's "Starting a Django 1.6 Project the Right Way" article.

Jeff has not written a guide for Django 1.7, but the two articles would be nearly identical.

In addition to using a virtual environment while developing, you will want to be able to simulate how your website performs in production (database and other services included). If you do not have a dedicated staging server, I recommend using a virtual machine instead. You can use VirtualBox to create as close a copy to your production server as possible. You can use Vagrant in conjunction with VirtualBox to automate testing on this virtual staging server. Additionally, if you're working on a larger team, ensuring your server provides continuous integration may be useful. Jenkins, Travis, Circle, and Codeship are all popular continuous integration options.

The Upgrade Process

Planning, selecting, and taking small steps is the name of this game. Consider yourself Caesar: you are here to divide and conquer. And divide and conquer you shall.

Like maintaining a formation in a Roman legion, upgrading Django is a strict procedure. In early September 2014, Aymeric Augustin laid out the practical steps to perform a major upgrade on the Django Developer mailing list. The following paraphrases his points:

1. Check and upgrade your project dependencies
2. Upgrade your project iteratively:
   1. Make a change according to the release notes
   2. Ensure the project test suite returns no errors
   3. Repeat

By splitting up the upgrade process into smaller steps, Aymeric has simplified the upgrade process for both major and minor upgrades.

In the following sections we will expand on Aymeric's points. We will first outline the actual upgrade process. We can then discuss how to properly plan for an upgrade and when to start performing upgrades.

This section will make it clear that you should only ever upgrade one release at a time. You should not upgrade from 1.4 to 1.7 directly. Instead, make the jump from 1.4 to 1.5 first, then proceed to 1.6, and so on. This is in keeping with the idea of taking small steps.

Upgrade Quirks

Iteratively upgrading from major release to major release by way of smaller, incremental project versions is desirable not only because it makes the upgrade process simpler but also because it may help avoid funny quirks.

Due to the introduction of custom user models in Django 1.5, the core development team opted to change the size of the URL encoder used during password resets. Prior to Django 1.6, password resets hashed user IDs into a base36 encoding. Starting in version 1.6, Django stared using base64 instead. To ease the process of upgrading, Django 1.6 came with a shim that would allow base36 URLs to continue to work. The shim only needed to work for a default of three days (or whatever the PASSWORD_RESET_TIMEOUT_DAYS setting had been overridden to). However, Django 1.7 does not have the shim. If you upgrade your site directly from Django 1.5 to Django 1.7, any of these password URLs will become invalid. It's a small issue, but one that could cause user frustration on a large site if developers skipped the 1.6 upgrade.

I have not spotted any other multi-version upgrade quirks, but that doesn't mean there aren't any. Smaller changes are key to identifying problems as they arise. Releasing smaller changes as incremental deployments may help avoid trickier upgrade problems like the one detailed above.

Handling Dependencies

Your project depends on more than just Django. You likely have a database, a few third-party Django apps, but also potentially message queues, short-term memory stores, caches, or a deployment system. Before you upgrade Django, you want to make sure that all your dependencies will still work together.

This section assumes you have a correct list of all of the dependencies for your website. If you do not, your first step is to figure out what your website needs to be deployed and run.

I recommend checking your Python version before you check any other dependencies. Django, your third-party apps, and any interface to external systems depend on specific versions of Python, making it a primary concern. If your new Django version requires a later Python version, you will want to upgrade Python before anything else.

The problem with my advice is that sometimes upgrading Python will require you to first upgrade other dependencies. For instance, pretend you are currently running Django 1.4 on Python 2.5 and you wish to upgrade to Django 1.5, which requires Python 2.6 or above. You thus first need to upgrade your Python version. The problem is that, as you examine your other dependencies, you discover your project is still running psycopg 2.0.5, released on September 1 2006, eighteen days before Python 2.6. Unfortunately, the legacy psycopg docs about supported Python versions are quite sparse, making your potential upgrade path unclear. However, psycopg's latest version, 2.5.4, supports both Python 2.5 and Python 2.6. The latest version of psycopg does not support your PostgreSQL 7.2 database (first released in 2002 and last updated in 2005 and deprecated in Django 1.3, to your increasing horror and panic). What's more, Django 1.5 requires PostgreSQL 8.2 and above. You thus must first upgrade to PostgreSQL 7.4 (supported by psycopg 2.0.5), then upgrade to to psycopg 2.5.4, upgrade to Python 2.6, and finally upgrade to PostgreSQL 8.2, all before even beginning your Django version upgrade. Remember that at each step of this process, you must run your test suite to make sure everything still works as expected.

If you try to deal with all of this information without structure, finding and acting on the information becomes a terrifying task. However, if you break it down into small steps, not only does the process become manageable, it becomes boring.

The difficulty with the process is not actually upgrading but figuring out what needs to be upgraded. Identifying each of the requirements of each dependency, via either the dependency's documentation or Google, becomes key. We can clarify the information by building a nested list in which each layer is an upgrade of a different dependency.

• Tasks to Upgrade Django
  • Requires Python 2.6
  • Tasks to Upgrade Python
    • Requires psycopg 2.5.4
    • Tasks to Upgrade psycopg
      • Requires PostgreSQL 7.4
      • Tasks to Upgrade PostgreSQL
        • All requirements met
        • Upgrade PostgreSQL->7.4
      • Upgrade psycopg->2.5.4
    • Upgrade Python->2.6
  • Requires PostgreSQL 8.2
  • Tasks to Upgrade PostgreSQL
    • Upgrade PostgreSQL->8.2
  • Upgrade Django->1.5

For each upgrade, we first identify the requirements for that upgrade, considering all of our dependencies. For any requirement the site does not meet, we then create another upgrade list. We then repeat the process (recursion!) until we reach a place where we only necessitate a single upgrade. We can then strip away everything except the upgrade instructions. This gives us a very simple to-do list:

1. Upgrade PostgreSQL 7.2->7.4
2. Upgrade psycopg 2.0.5->2.5.4
3. Upgrade Python 2.5->2.6
4. Upgrade PostgreSQL 7.4->8.2
5. Upgrade Django 1.4->1.5

If the nested list doesn't help you, you might prefer solving the problem with a graph. This actually allows you to reason about more intricate dependencies. For instance, the PostgreSQL 8.2 requirement actually depends on the upgrade to psycopg 2.5.4, a requirement we avoid in the nested list because it's already installed but that may be helpful to outline in other situations.

The use case above could be even more complicated than it is, especially if you do not plan and build a list as above. Imagine if psycopg 2.0.5 hadn't supported PostgreSQL 7.4. You would have had to first upgrade psycopg to a version that supported both PostgreSQL 7.2 and 7.4 in Python 2.5. Let's pretend that's psycopg 2.1. You would then upgrade PostgreSQL 7.4. Finally, you discover that psycopg 2.1 doesn't actually work in Python 2.6: Python 2.6 requires psycopg 2.5.4. This latest incompatibility would force you to upgrade psycopg again before being able to upgrade Python (if the necessity for PostgreSQL 8.2 didn't already). As you do this, you would of course need to pay attention to all your other dependencies as well. Finally, you would buy yourself an ash tray to accommodate your new chain-smoking habit.

You can avoid both the complication and the chain-smoking by simply building a nested list again.

• Tasks to Upgrade Django
  • Requires Python 2.6
  • Tasks to Upgrade Python
    • Requires psycopg 2.5.4
    • Tasks to Upgrade psycopg
      • Requires PostgreSQL 7.4
      • Tasks to Upgrade PostgreSQL
        • Requires psycopg 2.1
        • Tasks to Upgrade psycopg
          • All requirements met
          • Upgrade psycopg->2.1
        • Upgrade PostgreSQL->7.4
      • Upgrade psycopg->2.5.4
    • Upgrade Python->2.6
  • Requires PostgreSQL 8.2
  • Tasks to Upgrade PostgreSQL
    • Upgrade PostgreSQL->8.2
• Upgrade Django->1.5

Due to the recursive nature of the nested list, which forces us to evaluate the requirement of every upgrade, the complication of psycopg's partial upgrade becomes quite clear. Stripping away all the other information yields another very straightforward to-do list.

1. Upgrade psycopg 2.0.5->2.1
2. Upgrade PostgreSQL 7.2->7.4
3. Upgrade psycopg 2.1->2.5.4
4. Upgrade Python 2.5->2.6
5. Upgrade PostgreSQL 7.4->8.2
6. Upgrade Django 1.4->1.5

Once again, if a nested list is not to your liking, a graph might be.

Suddenly, the process is not only simple but quite unexciting. For each of the upgrade steps above, you must:

1. Upgrade the dependency
2. Test
   1. Run Test Suite
   2. Fix Any Problems
   3. Repeat Process until Correct
3. Update Documentation
4. Update Deploy Scripts
5. Commit and Push Changes in Version Control System
6. [Repeat for the Next Dependency!]

Suddenly there is no fear of unexpected dependency problems, as our strict procedure has ensured that all of our tools will work together at each upgrade step. You are moving your project from one valid deploy to another valid deploy in the smallest steps available. Your job for upgrading dependencies is to be a robot: follow the tasks above!

This process of small steps to valid states is what makes the process dull, repetitive, and very manageable. Imagine if we instead had upgraded straight to PostgreSQL 9.3. Our site, with psycopg 2.0.5, would be unable to run, putting us in an invalid state and forcing us to make larger, sweeping changes across the entire site, potentially leading to seemingly insurmountable problems or changes.

Not only do we want to make small changes, we want to make as few as possible. If Python 2.6 had not required psycopg 2.5.4, we would stop our upgrade of psycopg at version 2.1. Our interest is not currently in our dependencies. Our focus is on upgrading Django, and so we want to do the minimum amount of work to be able to move up to the next Django version. Our goal is thus to check our dependencies and do work only if necessary at this stage. Upgrading your dependencies is a separate process and should be performed regularly.

As alluded to in the list above, as you upgrade your dependencies, remember to update any files that deal with these upgrades. Updating requirements.txt, any deployment scripts, and project documentation as you go will make future work far less confusing.

What if you're unable to upgrade Django because you cannot upgrade a dependency (for one reason or another)? We will discuss this eventuality in the To Upgrade, or Not to Upgrade.

Upgrading Your Project

Once all of your dependencies work with your new version of Django, you can begin to upgrade your project. As with the rest of this process, we can split the task into different stages.

Planning

Your first objective is to understand the new version and to plan for the upgrade. Reading articles online may save you time by pointing out pitfalls, but the most important piece of literature is the release notes document for the version to which you are upgrading. Supplementing your reading with all the documentation in the Tools For Upgrading Django section is advisable.

Upgrading is like walking a tightrope. You want to be lazy and isolate only what you need to change to achieve an upgrade. At the same time, you cannot be too lazy, as it will force you to do more work down the line. Planning will help you figure out what changes you need to make and when to make these changes, as we will discuss shortly.

Django Package and Backwards-Incompatible Change

Upgrading Django itself is easy, as it simply involves telling pip to upgrade the Python package in your controlled environment and upgrading any documentation and deploy scripts. The tedious process of ensuring your test suite passes after performing this upgrade is the heart of the actual work. Many of the changes detailed in the "Backwards-Incompatible Changes" section of the release notes will now be raising errors.

Being greeted by a sea of errors is every developer's nightmare, and any action you can do to anticipate and manage errors in advance is in your interest. As such, understanding when to make changes to your code is crucial. Do you refactor code before or after you've upgraded the version of Django in your controlled environment?

For instance, when upgrading to Django 1.5, developers had to explicitly define the ALLOWED_HOSTS setting when DEBUG was set to False. It would be in your interest to plan to make this change before upgrading your environment: the setting works fine in version 1.4 and avoids a problem before you're inundated by others.

Just as the release notes help inform you about when to make changes, the documentation may help you decide how to make changes. Rather than deal with errors on a one-by-one basis, you may discover it is possible to fix all the issues in a single go. When upgrading to Django 1.5, it became mandatory for all url template tags to have view-identifier strings surrounded by quotation marks. You could use sed or your IDE's find-replace tool to change all the instances in a single command. This saves you an enormous amount of repetitive work. This is also a change you could make before upgrading the environment to Django 1.5. You could emulate the new url template tag syntax in Django 1.3 and 1.4 with the help of {% load url from future %}. This allows you to add this call to every template, modify all the url template tag calls, upgrade Django, and then remove the prefix.

Once you've upgraded your Django package and have a test suite that runs, you can deploy your site. This is the smallest set of changes required to reach your first upgrade goal.

Deprecated Features

Even though you have a stable, deployed site, you should not stop the upgrade process. You've dealt with all the changes listed under the "Backwards-Incompatible Changes" section, and now its time to deal with the "Features Deprecated" section.

Dealing with deprecated features generally involves swapping out parts of your project for new dependencies. For instance, if your Django 1.4 project uses the django.contrib.markdown app, your upgrade to Django 1.5 is going to require you to replace a substantial portion of the code in your project that handles markdown text. This is not as difficult as it sounds: There are three third-party apps dedicated to the task of replacing django.contrib.markdown:

1. django-markdown
2. django-markdown2
3. django-markdown deux

Each package provides a different set of tools, meaning that one might suit your purposes better than the others. Of course, you could also use Python-Markdown directly.

In the cases of Django-contributed apps like localflavor and comments, their full code and functionality persist as third-party app packages:

• localflavor
• comments

Other deprecated features typically involve changing APIs. In Django 1.5, calls to select_related() no longer accept the depth parameter, requiring simply that you remove any depth arguments in these calls. Similarly, in Django 1.6, the entire transaction API changed because of the new transaction system. In both cases, you could use grep to find all the instances of this code and then replace them by hand.

$ grep -nR 'select_related' project_dir | grep 'depth'

If you have an overwhelming number of changes to make, you could chain grep, cut, xargs, and sed to make all the changes for you. And just think of what will happen when you post your bash command online!

Your site doesn't need these changes right now, but you will need them in two versions. Taking the time to handle these features now will not only silence all the pesky warnings Django is raising but will also save you time two versions in the future. Future-You will thank Present-You for dealing with this now rather than later.

Unlike with the backwards-incompatible changes, you can deploy your site with every change you make in this section, as your site is fully functional at each change. You could also wait until you've replaced all the deprecated systems and APIs to deploy. In any case, remember to test as often as you can.

New Features

At this stage, we've removed all the incompatible and deprecated code in favor of functioning features. Integrating Django's new features into our project is the next and last step in a full upgrade process.

When you plan your upgrade, your goal is to isolate and focus on only what you needed. Waiting to integrate new features saves you time and makes upgrading easier but allows you the benefits of working in Django's latest version.

When upgrading to Django 1.4, it was possible to disable timezone support by setting USE_TZ to False in the project settings. Doing so while performing the last two upgrade steps enables developers to concentrate their efforts on work that must happen first. New features in Django should be treated as new features in your project when possible.

With that said, if at any point in the last two steps you could ease the transition by using a new feature, you absolutely should. You should immediately consider the use of QuerySet.prefetch_related() or ModelAdmin.save_related() if you find yourself rewriting a query, even if you're at the backwards-incompatible stage of an upgrade to Django 1.4 (Django 1.4 is when the features were introduced). This is why the planning stage is so crucial. Understanding that certain new features are unnecessary during the first upgrade steps and knowing when to integrate a new feature can significantly ease the upgrade process.

No matter when you integrate a new feature, remember to add tests as necessary and to deploy with each new feature (if possible). With this done, you will have fully updated your project to a new Django version. Remember to congratulate yourself and/or high-five your team.

When to Upgrade

The obvious time to upgrade is as soon as the new version is available.

This section assumes no problems when upgrading dependencies. We will discuss this eventuality in the To Upgrade, or Not to Upgrade section.

With minor releases, this is absolutely the case. Most minor releases require no change on your part and necessitate only an upgrade of the Django package in your environments and deployment scripts. You should, however, still consult the minor release notes and test your project on a staging server, as you would with a major release.

With a major release, you can actually preempt the first few steps to the upgrade process before the full release. Remember that the core development team creates a release candidate before releasing the full version. You may begin the upgrade process as soon as the release candidate for the newest version is released, but you should wait until the full release to deploy to your production (public) server. You'll be able to deal with your dependencies early on and potentially even deal with deprecated features before any others.

Providing you with the opportunity to divide and conquer early on is not the only advantage to starting with the release candidate. Another advantage is the potential to actively fix problems in both Django and your site. If you run into any problems or difficulties with the release candidate, it will allow you time to submit bug fixes to Django. This enables you to participate in Django's release process and potentially make the upgrade process easier for both you and others.

Conclusion

The steps provided in this section should act as guidelines for your own upgrade process. Every upgrade---both minor and major---is going to require slightly different steps, which is why you should always take the time to read about the upgrade and to plan for it, as it could save you enormous amounts of time. In short, however, you should aim to:

1. Check Dependencies and Upgrade as Needed
2. Upgrade Django
   1. Plan
   2. Preempt Backwards-Incompatible Changes
   3. Upgrade the Django Package
   4. Backwards-Incompatible Changes
   5. Replace Deprecated Features
   6. Integrate New Features

Remember to test, and aim to deploy as often as possible.

Upgrading Django to 1.7

Nothing helps understand theory better than a concrete example. Now that we understand the tools at our disposal and the process we will use them in, let's outline our upgrade to Django 1.7.

If you are upgrading to Django 1.7, you should currently be running Django 1.6.8 (or later, if another minor release of version 1.6 has occurred after the publication of this article) and upgrading to Django 1.7.1 (or later). Remember to test!

Project Dependencies

Your first step will be to upgrade your project dependencies. Django 1.7 requires Python version 2.7 or 3.2 and above. Ensure that your third-party apps support Django 1.7 and the Python version you're running and begin to upgrade these packages iteratively if they do not. Remember to plan all of the project dependency upgrades well in advance and to use nested lists to ease the process.

Your deployment method counts as a dependency, and you may need to upgrade how you serve Django. As of Django 1.7, the framework no longer supports FastCGI (not an easy decision). The FastCGI component in Django is only deprecated, and it won't be removed until Django 1.9. However, my opinion is that you should deal with this while still in Django 1.6. If your host only supplies FastCGI access, I highly recommend finding a new server that will allow you to migrate to a WSGI configuration.

The upgrade to Django 1.7 makes dealing with South a bit odd. South does not run in Django 1.7. At this stage, you should thus focus on other dependencies and plan to deal with South during the backwards-incompatible upgrade or new features stages, as we shall discuss.

Plan and Preempt

If you have not already done so, read through the Django 1.7 release notes. Identify anything you might be able to do before actually upgrading Django in your environment. For instance, you might need to integrate pytz and can easily do so before going any further.

Also take the time to identify new features you may be able to use as you refactor code. As you refactor incompatible or deprecated code you may be able to replace some raw SQL with a Prefetch or Lookup Object or to use the new QuerySet.as_manager() method to simplify your models.py files.

Backwards-Incompatible Changes

You can now upgrade Django in your environment and begin to work through all of the changes in the backwards-incompatible changes section. Remember that you should not only be testing as you go but that you now also have access to Django's own system check framework. Pay close attention to any output the checks framework might give you!

If you were using South in Django 1.6, the first thing you will want to do is re-enable migrations. Move migrations/ to south_migrations/, and create a migrations/ directory with an empty __init__.py file inside. This allows you to create a first native migration by invoking $ ./manage.py makemigrations. Recall the existence of ticket 22608, mentioned in Part III.

If you were not using South previously, then you should treat migrations as a new feature and ignore them for the time being. Recall that failing to have a migrations package in any app will result in legacy behavior (no migrations). The exception is if your app has a relationship to an app that is migrated, such as any in the contributed library. If your app has a ForeignKey to the auth.User model, then you cannot treat native migrations as a new feature and must begin using them now.

If you are upgrading an app, rather than a project, recall that is is possible to have both a south_migrations and a migrations directory in the app and to run the two side by side, allowing for apps that are compatible with Django 1.6 and 1.7. Part III of this article series has more details.

Walking item by item through the changes documented will ensure that you don't use initial data and that your custom fields have a deconstruct() method. A pitfall to watch for is import errors caused by the new app registry. As mentioned in the last article, the app registry must be built before certain operations like translation or loading the custom user model may take place. This may force you to change some of your import sequences. This is detailed in the documentation about the app registry.

Test and deploy and cheer! You're now running Django 1.7.

Deprecated Features

Even though you have a stable, deployed site, you should not stop the upgrade process. You should now remove deprecated features from the project. These features will disappear for good in Django 1.9. A lot of these changes are straightforward, if monotonous.

For instance, you'll need to replace any instance of the IPAddressField with the GenericIPAddressField. In this case, you will need not only to run your tests but also to create a migration file (if you have the native migration system currently working), as you are changing your models. Remember that the command line is your friend for changes like this.

Removing all the deprecation warnings prepares you up for upgrading the next two versions of Django with far less effort.

New Features

New features are the last milestone. In Django 1.7, there are a lot of powerful new features that will shorten your code and improve your site.

If you do not have native migrations yet, this should be your first job. Create a migrations/ directory with an empty __init__.py file inside. This allows you to create a first native migration by invoking $ ./manage.py makemigrations.

Seriously consider using the new system check framework to verify key pieces of your project, as checks may be used to avoid issues early in development and right before deployment.

Conclusion

Upgrading Django to version 1.7 is a great demonstration of the tools and process discussed. It serves not only to reinforce the outlined process but also to remind you that your should treat it as a guideline: We could not deal with our South dependency until the backwards-incompatible stage.

I'm sure you're sick of hearing it, but remember to test as you go. The ideal, once again, is to make small changes, improving your site incrementally from one valid deploy to another valid deploy. Focus on each smaller milestone rather than the ultimate purpose, and remember that even if you cannot perform a full upgrade, taking any of these steps will help you in the long run.

To Upgrade, or Not to Upgrade

Spoiler Alert: Upgrade.

If you are not using a supported version (1.7.1, 1.6.8, 1.4.16), the time to upgrade is now. It is not completely unreasonable to still be on Django 1.5.10, as 1.7 was released on September 2, 2014, but you should be planning your upgrade to 1.6.8 for before the end of the year. I further argue that you want to be running Django 1.7 as soon as possible. If you are still running Django 1.4, note that LTS support is slated to end in September 2015 (more precisely: 6 months after the release of Django 1.8).

In the following sections, I will address arguments against upgrading and hopefully convince you that upgrading should be done regularly.

Time and Cost Arguments against Upgrading

Gentlemen, you can't fight in here! This is the War Room!

Organizations will often argue that they do not have time to "chase releases" or that internal sites do not need to be upgraded because they are private. Some developers have argued that upgrading is an expensive process in terms of both time and money and that a project needs to "just work." Others still would prefer to focus on new features and production and see upgrading as a disruption.

The core of the issue is that none of these arguments account for security, technical debt, or access to community support.

Django has seen numerous releases for security reasons, ensuring that (at the moment), Django 1.7.1, 1.6.8 and 1.4.16 are all as safe as the development team can make them. However, security fixes are only applied to supported versions. If you are running a version that is no longer supported, you are vulnerable to at least one attack and probably more.

Furthermore, if you are not running the latest set of releases, you are probably incurring technical debt. As you modify your website, you and/or your team will make changes that make sense for your current version of Django. These changes may not make sense for later versions, meaning you are writing code that you will have to change and that you could write differently (or avoid entirely). The longer you stay in an earlier version of Django, the more you incur technical debt and the more difficult and time consuming your future upgrades will be. This is not only an argument against using unsupported versions but also against staying with LTS versions for prolonged periods without reviewing and planning for the changes in new releases.

Avoiding technical debt is also why planning an upgrade is so important. If you make a change in the backwards-incompatible change step, only to replace it with a new feature later on, then you have made two changes to your code where you could have made only one. The principal in the long run is the same but typically grows much more complicated.

As discussed in Part I, Django 2.0 was originally slated to be a new major release in the semantic versioning sense, breaking backwards compatibility and introducing a slew of new features and improvements. This is no longer the case, with Django 2.0 simply being Django 1.9's successor. With the official change of the versioning system in Django 1.7, Django is currently poised to continue releasing regularly with scheduled deprecations. This reinforces the argument for maintaining an up-to-date Django project, as no release will allow you to catch up in a single upgrade once you've fallen behind.

Finally, if you are not running a supported Django version, you lose community support. I'm not saying people will not help you: you will get help if you email the Django user mailing list to request help on Django 1.1, but only from those who are using or have used that version. The number of people using the latest supported versions is much larger, and so the help available is much greater. The same can be said for third-party apps. Third-party apps typically only support active Django versions. It is possible to use older versions of these apps, but, again, this may make you vulnerable to security issues and will prevent you from using useful new features.

If none of this convinces you that upgrades are worth the time, consider upgrades in a different light. By investing a small amount of time and effort now, you gain about nine months worth of work from all the Django contributors. Even if the upgrade takes a single developer a whole month (and it should take nowhere near that), the return on investment for the upgrade is at least nine times that work.

Dependencies Should Not Block Upgrades

The last argument for not upgrading a major release is that the dependencies cannot be upgraded. This presents itself either as the inability to upgrade Python or project reliance on an unmaintained app.

The inability to move from Python 2.x to 2.7 is typically not a technology problem but an institutional problem. I cannot advise you on how to handle such a problem (as each one will be slightly different), but I can point out that Python 2.6.9, released October 2013, was the last security release for Python 2.6. All the security issues dealt with in Python 2.7.7 and Python 2.7.8 are potential vulnerabilities in Python 2.6 (not to mention the enormous list of bugs). Most notably, the security additions of PEP 466, to be fully implemented in Python 2.7.9, will not be backported to Python 2.6 (Python 3 already had these changes). Your project should---at a minimum---be running off of Python 2.7.

If your upgrade appears blocked by a third-party app dependency, you may still take steps to upgrade your project.

If the app is being maintained, you could simply wait until the maintainers update the package. Instead, consider investing the time to help the package maintainers update the app to work with the Django version to which you wish to upgrade your project.

Typically, however, apps acting as upgrade blockers are not maintained. Due to security and bug-fix concerns it is not in your interest to use unmaintained packages. Failing to upgrade and keeping an unmaintained app forces you to incur even more technical debt than discussed before. You have two choices to move forward. You may either look for another app that meets your requirements and switch dependencies or else begin to maintain the app yourself.

Actively contributing to a project is not only in your interest but also in your organization's interest. By investing the time to maintain an app, developers across the globe (and potentially other organizations) will contribute to the code and test the app for you. Furthermore, you'll feel good about it.

Python 3

No article about upgrading would be complete without mentioning Python 3. Django 1.6 heralded official support of Python 3, providing a full set of upgrade instructions, and key parts of the community have been vocal about its adoption. Jacob Kaplan-Moss gave talks about upgrading apps to Python 3 at both PyCon US 2013 and DjangoCon AU 2013, and Aymeric Augustin wrote on his indiegogo fundraiser that he "worked on Python 3 because [he] couldn't stand Python 2 any more."

Despite the fact that Python 2.7 will now be supported until 2020 (instead of 2015), I would urge you to plan now for an upgrade to Python 3. While Django will support Python 2.7 until that time, the benefits of moving to Python 3.4 are enormous. The Python 3.4 interpreter is faster, more memory efficient, and more feature-full than 2.7. Given the discussion of PEP 446, you also know that Python 3.4 is more modern and the focus of the Python development team. It should be your focus, too.

In Review: Upgrade, Upgrade, Upgrade

Django is 128,000 lines of code you do not have to actively manage but that you benefit from directly. Between the release of Django 1.6 and Django 1.7, over 300 developers have contributed directly to Django's codebase, resulting in hundreds of person-hours of work invested in the next version of Django. These changes include bug fixes, security updates, and new features. By investing a small amount of time on your own project, you gain the power and additions of the new Django version and avoid incurring technical debt. Even if you end up investing time in helping or maintaining an open-source, third-party app, the return on investment of staying up-to-date is enormous. Upgrading Django should be at least as large a priority as new features.

The actual number of developers who have worked on the codebase between the release of Django 1.6 and 1.7 may be computed with the following command:

$ git log \
    --since={2013-11-06} \
    --before={2014-09-02} \
    --no-merges \
    --format="%aN" \
    | sort -u \
    | wc -l
331

Consider that this is a fraction of the work: it does not include administrative work, discussing changes on the mailing list, and the various patches that people iterated through on the bug tracker.

Django's Future

As you develop and upgrade Django, it is useful to consider changes currently being made to Django's codebase as well as planned changes to the web framework. In the following sections, we will look at Django's release cycle, future features to expect, and examine how to preempt changes in Django.

Django Release Cycle

Django's release cycle appears to be speeding up, to the joy of some and the frustration of others. The original roadmap for Django 1.7 scheduled the release of the latest version for May 15, 2014. If that date had been met, the time between the releases of Django 1.6 and Django 1.7 would have been only 190 days, which is far shorter than previous releases. The average number of days between major releases (since 1.0) is 313 with a standard deviation of 34. You can distinctly see a decline in the graph below.

Django 1.7 was actually released on September 2, 2014, which means Django 1.6 spent 300 days as the latest version.

The trend is far less clear and, following the coefficient of determination, also far less reliable. Whether the release cycle is actually getting shorter is thus unclear.

Given the average of 313 days between major releases, my attempt to channel the spirit of Nostradamus would lead me to tell you to expect Django's next release in July of 2015. However, Tim Graham, the release manager for Django 1.8 and the Django Software Foundation's current Django Fellow has slated Django 1.8's release for March 30, 2015, as noted in Django 1.8's Roadmap. This would leave Django 1.7 as the latest version for 209 days, three standard deviations from the current average. To best try to meet this ambitious release date, Tim has opted to freeze features in alpha rather than beta, in hopes this will allow for earlier testing periods..

While I think changing when features are frozen is a good method to keep the release process on schedule, I would be surprised if everything ran smoothly and personally expect Django 1.8 in May or June (one or two standard deviation under average). This is, of course, pure speculation. However, Django's actual release date is somewhat beside the point. The heart of the matter is that you or your organization should be prepared to upgrade Django to version 1.8 in August or September of 2015.

Django 1.4 LTS was slated to end in March of 2015. As Django 1.8 is expected to become the next version to receive LTS, the core development team has pushed Django 1.4 LTS back to September 2015, or six months after Django 1.8's final release.

Preempting Change: patterns()

This section features (heavily modified) excerpts from my book, Django Unleashed, currently in "Rough Cut" (draft) form. Please consider taking a look on Safari.

Thanks to the release notes and the deprecation timeline, it is possible to preempt changes in future Django versions. In this section I want to discuss a single change expected in Django 1.8.

The Django 1.8 release notes detail the removal of patterns(). You don't actually need to wait for Django 1.8 to remove patterns() from your projects, however. The call to patterns() is unnecessary. The final result is just a list of RegexURLPattern objects, which we can create manually via calls to url(). We could quite easily define urlpatterns as:

from camelot.views import knight_list, knight_detail
...
urlpatterns = [
    url(r'^knight/$',
        knight_list,
        name='roundtable_knight_list'),
    url(r'^knight/(?P<name>[\w\-]+)/$',
        knight_detail,
        name='roundtable_knight_detail'),
]

Taking the small step to replace patterns() with a list any time you edit a URL Configuration in your project can save you time at the next upgrade. Keeping an eye on the next releases' roadmap, release notes, and the Django deprecation timeline may also reveal other useful changes you can make as you go about building your project, preempting code debt.

Future Features and Deprecations

Core developers have publicly committed to "slim down" Django's contributed app library. This may be a little misleading, however. The goal is not to remove the contributed library but instead to ensure that Django provides only two things:

1. Tools necessary for every dynamic backend, such as HTTP handling, an ORM, and migrations
2. Features that are difficult to program but identical on many websites, such as the authentication app

The webdesign app is thus unsurprisingly scheduled to be removed in Django 1.8 (the lorem template tag will become a regular tag). However, as per Marc Tamlyn's kickstarter in March 2014, we can expect a new contributed app to extend PostgreSQL support in Django 1.8. You can read all about it on Marc's blog as he builds the new app.

Following in the footsteps of Andrew Godwin and Marc Tamlyn, Aymeric Augustin crowd-funded the refactoring of Django's template system as well as the addition of an API that would allow Django to easily integrate with third-party template engines such as Jinja2. The indiegogo fundraiser met its initial target in just two days, followed by the only stretch goal on October 8. Aymeric has already begun work on the new template system and will be maintaining a blog about the work he does for the project.

Article Conclusion

To quote Mark Lavin's own upgrade article: "Tests are a Must." Once you have tests, use the release notes and deprecation timeline and then divide and conquer the upgrade process. Small boring steps will also help you avoid replicating Monty Python's Silly Walk. Even if you cannot perform a full upgrade, by making minuscule monotonous modifications you make your life easy and uneventful, and you help any future work on the project by avoiding technical debt and conserving security. Keep in mind any changes that affect documentation or deployment, and make those changes immediately.

Upgrading is not only about the process of upgrading Django but also about being prepared to do so. Making sure you or your organization have scheduled the time to upgrade is crucial. You should also schedule maintenance and upgrades for any site dependencies.

Compared to the last two upgrades, the process for upgrading Django to version 1.7 is a little more involved. Adhering to a clear, simple methodology is even more important than before.

Series Conclusion

Django 1.7 is an exciting new version, but fully grasping all that it is capable of requires a significant time investment. I hope this article series helps make Django 1.7 more accessible to you. If there are any sections you find unclear, please let me know on the Django User Mailing List. There is a thread specifically for this article series.

I was only able to write this because of Django's precise and well-maintained documentation and because many of the developers in charge of the new version took the time to document their work. Special thanks to Andrew Godwin, Aymeric Augustin, Christopher Medrela, and Russell Keith-Magee for writing about their work.

Last but certainly not least, thanks to Jacinda Shelly and Amy Bekkerman for all their work improving and clarifying this content.

Series Notes

• Upgrading Django Checklist
• Article Series Links