Django, PostgreSQL, and Transaction Management

Django has quite a bit of code in it devoted to transaction management. Although the documentation goes into quite a bit of depth on transactions, I’ve never felt that the docs by themselves let you build a good mental model of how transactions actually work. So, I decided to approach it experimentally: Build a small Django app, and see how the various options actually work.

For all of this, I’m using Django 1.1.1 and PostgreSQL 8.4.1, through the psycopg2 interface.

For the application, we have a models.py:

1
from django.db import models
2

3
class Address(models.Model):
4
    street_address = models.CharField(max_length=80)
5
    city = models.CharField(max_length=80)
6
    state = models.CharField(max_length=2)
7
    zip = models.CharField(max_length=5)
8

9
    def __unicode__(self):
10
        return self.street_address + " " + self.city + " " + self.state + " " + self.zip
11

12
class Order(models.Model):
13
    customer_name = models.CharField(max_length=80)
14
    shipping_address = models.ForeignKey(Address)
15

16
    def __unicode__(self):
17
        return "Order " + unicode(self.id) + " going to " + \
18
            self.customer_name + ", " + unicode(self.shipping_address)

and a views.py:

1
from django.http import HttpResponse
2
from djangotrans.example.models import Order, Address
3
from django.db import transaction, connection
4

5
def create_order(request):
6

7
    address = Address(street_address="1112 E Broad St", city="Westfield", state="NJ", zip="07090")
8
    address.save()
9

10
    order = Order(customer_name="Gomez Addams", shipping_address=address)
11
    order.save()
12

13
    return HttpResponse("Created: " + unicode(order))
14

15

16
def show_order(request, order_id):
17
    order = Order.objects.get(pk=order_id)
18

19
    return HttpResponse(unicode(order))

and a urls.py:

1
from django.conf.urls.defaults import *
2

3
urlpatterns = patterns('djangotrans.example.views',
4
    (r'^createorder/$', 'create_order'),
5
    (r'^showorder/(?P<order_id>\d+)$', 'show_order'),
6

7
)

The Basic Rule

In absence of any mention of transaction management at all, Django opens a new transaction with each database operation, and closes it when the operation is complete. If all you do is read operations on the model, it does a ROLLBACK to close the transaction; if you do a data-changing operation, it does a COMMIT. For example, in create_order, the pattern looks something like:

1
# BEGIN
2
address = Address(street_address="1112 E Broad St", city="Westfield", state="NJ", zip="07090")
3
address.save()
4
# COMMIT
5

6
# BEGIN
7
order = Order(customer_name="Gomez Addams", shipping_address=address)
8
order.save()
9
# COMMIT

… and in show_order:

1
# BEGIN
2
order = Order.objects.get(pk=order_id)
3
# ROLLBACK

This is simple, straight-forward… and wrong. The two operations in save_order should be wrapped in a single transaction. If we saved the address, but didn’t then save the order, we’d have an “orphan” address, and that would be just tragic. So, what to do? Fortunately, Django provides a rather large toolbox full of other ways of managing transactions.

Transaction Middleware

First, let’s take a look at the transaction middleware. This is a middleware component that you add to settings.py:

1
MIDDLEWARE_CLASSES = (
2
    'django.middleware.common.CommonMiddleware',
3
    'django.contrib.sessions.middleware.SessionMiddleware',
4
    'django.contrib.auth.middleware.AuthenticationMiddleware',
5
    'django.middleware.transaction.TransactionMiddleware',
6
)

This component sets a flag in Django turning off the automatic COMMIT/ROLLBACK behavior, and instead does an automatic BEGIN when each request starts, and then a COMMIT (if the view function returns normally and did a data-changing operation) or ROLLBACK (when the view function throws an exception, or returns normally having not modified data.)

Doing this to our extremely sophisticated application above, we get a much more pleasing pattern of transaction management:

1
# BEGIN
2
address = Address(street_address="1112 E Broad St", city="Westfield", state="NJ", zip="07090")
3
address.save()
4

5
order = Order(customer_name="Gomez Addams", shipping_address=address)
6
order.save()
7
# COMMIT

… and in show_order:

1
# BEGIN
2
order = Order.objects.get(pk=order_id)
3
# ROLLBACK

Much better.

Transaction Decorators

Django provides a set of transaction decorators that can be added to view functions to explicitly control how transactions work for that particular function. They work the same way whether or not the transaction middleware is present. They are:

• @transaction.autocommit – This sets the default Django behavior: A COMMIT after each save(), and a ROLLBACK after each database read. If the transaction middleware isn’t present, this doesn’t do anything; if the transaction middleware is present, this reverts this particular function back to the default behavior.
• @transaction.commit_on_success – This sets the transaction middleware behavior of commiting all of the work done in a single view method (if data modification occurs and the view returns successfully), or doing a single ROLLBACK at the end (if data modification does not occur, or the method throws an exception). It’s redunant if the transaction middleware is enabled.
• @transaction.commit_manually – This overrides the current transaction behavior, and requires that the view function call either transaction.commit() or transaction.rollback() at the appropriate place or places (they can be called multiple times within a single view function). If a view function exits with data-modifying work done after the last commit() or rollback() call, the decorator will throw a TransactionManagementError exception.

So, What’s Not to Like?

That seems to be a pretty comprehensive set of transaction control mechanisms. So, what else could one want? Let’s take a look at what is actually sent to the database when we show an order:

1
LOG:  statement: SET DATESTYLE TO 'ISO'
2
LOG:  statement: SHOW client_encoding
3
LOG:  statement: SHOW default_transaction_isolation
4
LOG:  statement: BEGIN; SET TRANSACTION ISOLATION LEVEL READ COMMITTED
5
LOG:  statement: SET TIME ZONE E'America/Los_Angeles'
6
LOG:  statement: SELECT "example_order"."id", "example_order"."customer_name", "example_order"."shipping_address_id" FROM "example_order" WHERE "example_order"."id" = 47 
7
LOG:  statement: SELECT "example_address"."id", "example_address"."street_address", "example_address"."city", "example_address"."state", "example_address"."zip" FROM "example_address" WHERE "example_address"."id" = 79 
8
LOG:  statement: ROLLBACK

The transaction here isn’t actually required at all, setting aside the other overhead. Can we get rid of it? In part two, we’ll see how we can.