Django Testing Gotchas

I recently started using Django's newly introduced capability to make use of test fixtures. In order to do this all test classes that need fixtures have to be subclasses of Django's custom TestCase class rather than the customary unittest.TestCase. I had written about 17 or so test cases split across two test classes. One of these test classes used Django's fixtures.

Each Django test run (kicked off by running python manage.py test) involves the following steps:

1. Create test database
2. Create the necessary tables
3. Load "initial fixtures" if any. These have to be located in a file named initial_data with suitable extension denoting fixture type (.xml, .json etc.)
4. Load fixtures specified in each test case
5. Execute any one test
6. Remove fixtures loaded in step #4
7. Repeat steps 4 through 6 until all tests have been executed.

Sounds nifty, doesn't it? That is what I thought too, until I ran into reality. All goes well until step 3. After that, the tests take an eternity to execute. OK, it took 159 seconds to execute 17 puny tests. I was surprised. My application was rather simple, involving less than a dozen tables and a few hundreds of rows. And I was only using a subset of the data to test. What was going on? I looked around and found that I was not the only one to face this problem.

After reading up more about the problem I figured that I could probably cut down on the time taken by "refactoring" my tests and moving unnecessary fixtures out to "initial fixtures" so that they would only be loaded once. Luckily this would work in my case as my app only used the data for reference; it did not modify any. I was therefore not constrained to load the data before each test.

I made the necessary changes and re-ran my test cases. The time taken to execute the test cases improved ever so slightly. It now took about ~140 seconds.

I was still unhappy with the results. Since I was not using any fixtures, I decided to substitute Django's TestCase with unittest.TestCase. And voilà! It was as I had engaged hyper drive. 24 tests now took a mere 0.2 seconds to run!

Naturally the question that came to my mind was that in the absence of fixtures, what on earth was taking up so much time? Django is known for its speed vis a vis Rails; I had come to expect the same kind of superior performance while running tests too. I don't have an answer right now. I hope to come up with one after I take a look at Django's innards.