Django Diaries / 27th Nov 2013

New Apps & Migrations

One of the early questions I ran into when designing migrations was how to make the onboarding experience for users as easy as possible. All new apps that you made in 1.7 should come with migrations from the start, of course, but we didn't want to automatically convert apps that didn't have migrations - those could be third-party apps or, as I found to my amusement during initial testing, django.contrib apps.

Several potential solutions were thought of and discussed between some of the core team and some South users I talk to regularly. The first thought was a new setting - perhaps MIGRATED_APPS - that defined which apps had migrations, but this was a bad idea and required people to opt-in - a definite no-no.

The second idea was to automatically exclude django.contrib apps and then have a blacklist setting - UNMIGRATED_APPS. As we're trying not to introduce more settings in Django, this wasn't looking great from the start, and this still needed makemigrations to prompt you for every new app and ask if you wanted migrations - asking every time if you said no.

I briefly considered making makemigrations write values to the settings file, but then I stopped being crazy and instead decided it could output the new value of UNMIGRATED_APPS to the console when it had finished a run - not very nice from a user experience perspective, but it worked and didn't stomp all over your existing apps.

Then another core developer - unfortunately, I forget who - made the brillant suggestion of shipping an empty migrations module in the 1.7 app template and then just having makemigrations automatically add an initial migration to apps with an empty migrations module. A little bit of extra code to ignore South-style migration modules (they are, unfortunately, named the same) and we have the new solution.

It's so easy to join!

This means, if you make your app using startapp in 1.7, then as soon as you add models and run makemigrations it'll just detect your models and make your first migration - simple!

If you have an app from a previous Django version, you can just pass it as an argument to makemigrations to force an initial migration to be created:

./manage.py makemigrations my_old_app

Because initial migrations auto-apply themselves if the tables they refer to already exist, there's no extra step; the first time you run migrate your new initial migration will apply itself and you're done. This is also how we'll ship initial migrations for contrib apps in 1.7; they'll be present, and they'll be used for new installs, but upgrades to existing projects will just have them automatically marked as applied.

GIS

Work is also underway on GIS support, one of the last major features to be tackled (the only remaining one is correctly handling the other ends of ForeignKeys changing - especially if it's a swapped model, i.e. auth.User).

GeometryField now serialises correctly, and the PostGIS backend has started gaining a Schema Editor which allows migrations to be done against GIS fields. It's not yet complete - it only does model addition, and not things like field addition or field alteration - but it's a start.

Documentation

Another big push I'm doing now is writing up some better documentation for migrations. This is quite a large task; as well as having to update the Field documentation and the signals and command references, there are numerous workflow examples all throughout the docs that refer to syncdb.

The biggest of these is the tutorial, which I plan to rework soon to use migrate rather than syncdb, and potentially introduce a new section that covers migration management.

1.7

This is all in preparation for the 1.7 release, which now 1.6 is out looms on the horizon. We're drawing up a release schedule now, but the plan is for a prompt, 6-month release cycle - putting the first alpha release soon after the new year, and the final release in the middle of 2014. We'll announce an official schedule soon.

Finally, I want to encourage people to contribute to migrations - both by filing bugs and also using direct patches. The dynamic around me working on this based on Kickstarter is an unusual one, and I want to make sure it doesn't hinder contribution efforts - if you think I'm wrong, please tell me. If you want to change something small, just make a patch.

At some point during the 1.7 release cycle I'll be calling the migrations project finished, and moving onto release management for the whole of Django; South 2 will continue to primarily be run by me, but beyond that point I'll be considering my Kickstarter work on Django complete.

I will, of course, continue contributing - it would be hard to stop me - but once again as a volunteer, rather than as a paid Kickstarter project. I suspect, however, that I will be buying my backers drinks for years to come!