Article / 2nd Jun 2010

On Django And Migrations

For at least a year now, people have been suggesting to me that South should be in Django core.

I've always resisted, for good reason - not only because of South's relative immaturity at the time, but also because forcing a single solution would, I think, not be a good thing in general. (South, admittedly, has a few design issues, but it's all about tradeoffs.)

However, last week at djangocon.eu, I revealed, in rapid-fire English (apologies to non-native speakers for that, we're working on time dilation) my Grand Plan for the future of South and Django migrations in general, one that seems to have the approval of at least a few members of Django core.

The proposal is not, as some expected, to put South completely into Django core; instead, South will be split into two parts. One, the part that implements database abstraction, dependency resolution, and history tracking, would be put into Django itself. The other part - autodetection, model freezing, and so forth, will remain as South.

(If you'd like more in-depth details of the split, see my post to django-developers.)

The idea behind this is simple - take the parts of South that nearly any migration solution would need (database abstraction especially), and move those into a place where they can be used easily by any migrations solution.

More interestingly, because the proposal describes a common idea of dependencies and of what a migration actually is, it means that migrations can be written such that they need no supporting framework at all, and can be just pure Python calls to the database APIs (if Django ever ships any migrations, this is probably how they'd be done), raw SQL files (with a version per database backend if needs be), or a Python file managed by a supporting library (South 2.0 migrations), and that these different formats can all coexist, even on the same application.

Some might argue that such a solution is slightly restrictive, but I see it as the job of a framework to provide some level of consistency, and by doing this, it means reusable apps can ship with migrations written using the library of their choice (or no supporting library, as the case may be).

What does this mean for South? Well, in the short term, South 1.0 will be released, being mainly a polishing-off release of 0.7, and introducing a few new concepts (like the "rebase"). In the longer term, if my proposal meets with acceptance from the community (but, more importantly, those illustrious figures that make up Django's core developers), I'll start work on a branch to move the relevant parts over and firm up some APIs, and it can hopefully be finished and production-ready in time for 1.4.

At the same time, I'll be cleaning up the South codebase to match, and will then release South 2.0 at the same time as the migration APIs are released with Django. South 2.0 will, necessarily, only support that version of Django - previous versions will have to stick with South 1.0, which will get maintenance fixes. I'll also ensure there's a reasonably clean upgrade path from 1.0 to 2.0.

So, I'd love to hear your feedback and thoughts on my proposal. The mailing list thread is probably a good place for constructive comments or criticism.

(Also, slides for my talk are available here, and there's also a video.)